I think the biggest component to whether something is seen as taboo or controversial is your perception of it from the start. If you're against something in real life - and I mean, like, gung-ho "Kill every last one of them N****rs" - you'll find the idea of interracial couples nauseating and very disturbing.
Personally, I find extremely overweight / very-pregnant women (basically, they look like blimps) and lolis to fall into this category. I just have zero interest in banging a chick who could suffocate me with her ass or is too young to understand what she's really doing, and feel slightly disgusted when I see it in games (even if everything's consensual).
Now, if you're not so against those topics, the idea of it in a game might not appeal to you, but you won't feel any revulsion or opposition to it being there. I don't care if incest is a thing, because I've read enough fantasy/fiction and real-world history to know that it's just something that happened, usually under the pretense of blood-line purity. Likewise, since I've never owned, seen, or spoken to a slave that wasn't into BDSM, I don't see the problem with that, either.
Where things really start to fall into the "morally grey" area is when you introduce some factor that makes it the NPC's choice to give in to the player's advances. Be it an overwhelming Charisma score, pheromones, hypnosis/mind control, huge muscles / breasts, or exploiting some past shame for personal gain, the idea remains the same that it's not YOU making the choice to do it, now - it's them. You're just along for the ride, happy to have it happen.
What I'd like to see in more games/VNs going forward here is the introduction of NPCs who realize that they've fallen victim to one of these factors, but is now so far gone and has changed so much that they rebel and try to fight back, even if they ultimately fail. What would also appeal to me is their realization that this was done to them, but their lives are now better for it, and wish to see others feel/live/act/behave in a similar fashion.
This means you can have sequels to games where the harem breaks apart, only to coalesce into a love story, or where it grows even bigger, or it falls apart entirely, or now the protagonist from the first game is now the big-bad-boss of the second/third, etc. These kinds of games will allow us to experience not only the initial mindset of the taboo - it's wrong, but it feels so right - and come out the other side with a new perspective, whatever that may be.