What is your experience in the swinging lifestyle? How many years? How many swingers (old school term), ENM (ethical non-monogamous which is the more modern term) couples and/or poly groups have you known well? Unless your sample size approaches at least 20, then I respectfully say that you're the one being self-righteous by claiming to speak for a group of people you don't know and don't understand. My wife and I have been married for more than 40 years and been swinging longer than that. We've met hundreds of people over those years and have had long term friendships with scores of couples. Please don't lecture me on my first hand experience.
Mindset IS absolute immunity if the people involved decide to make it so. It depends on discipline, morals, training, opportunity, and rational choice. If you don't think this is a true statement, I strongly suggest that you find a BDSM club and talk to a D/s couple. If you tell THEM mindset is frail and can break down they will laugh in your face. The choice to ignore the opportunity is pretty much the basis of their entire relationship.
People who choose to live this lifestyle usually have long discussions with their partners beforehand about boundaries and limits. Those that don't have those discussions DO turn into the cheaters. In the long run, those usually aren't healthy relationships.