Funnily enough, the only people I knew openly swinging were around 30 years old and below, all younger to way younger than me. I guess it is a bit iffy to associate kinks with certain ages.
Apart from that the general problem with Swinging for many people is that it is too close to NTR for their likes. If you dislike these kinks, the main difference would be somebody getting it on with MC's LI with your consent instead of without MC's consent. Which would not be helping if it were without "the player's consent" so to speak. At the same time those who like NTR may not find mere Swinging quite as fulfilling, feeling like it lacks the punch. Of course there is always the thing that not everybody who wants a kink wants to test it to the extreme (e. g. for BDSM: a bit of spanking is okay, as are dom/sub relationships, but leave it in the bedroom and the whip completely out of it) and this game has been criticised already for "turning it down too far". Then again, NTR seems to be one kink where many are pretty set in their attitude towards it, so it may not hold here.
Which is of course more a general opinion about why swinging is so rare - for this game it doesn't matter as much because
(a) everything is mostly optional (I say "mostly" because if it's like week 1 and you need to play a certain amount of kinks, you may have to turn one in - if you're only thinking about one specific kink, it probably can be avoided completely), and
(b) NTR is already announced, so this "competition" between the kinks falls flat.