I guess that's always the compromise a dev has to make when designing their game.
You can make a game where you have complete freedom to choose the LI's that you want to and choose all of the scenes and kinks and whatever precisely the way that you want to but then the story is probably not gonna make a lot of sense and it's going to seem just like a bunch of mashed-up, non-sensical sex scenes.
Or you could have a game that's just completely linear, you don't get any say whatsoever where the story, and therefore the relationships, go and you're just along for the ride. Here, the story could potentially be pretty good because the dev is just free to write what they think is sensible, but people are obviously gonna be pissed that they have no choice whatsoever in a supposed "game".
Most games on this site try to straddle that line and fall somewhere on the spectrum in between these two options but I guess you'll annoy some people with your choice no matter what you do because we all have a preference here. I mostly prefer the more linear storytelling like with WVM but yeah, it's just a choice and a preference.