I think the main problem in this case is that a single developer (or too small developer team) creates too many characters for him to handle. Too many single developers want to make a game which requires a team of several people. In short: they are overambitious.
Also, I have an impression that many underestimate how much time and resources freedom of choice and non-linear plot costs. Not to mention that freedom very often stays in opposition to the quality of the story. In order to write a good story with believable character progression, a writer needs to control the order of the events within a story, which of course stays in opposition to freedom of choice. I don't understand why many players treat non-linearity as a holy grail and consider linearity and single ending a flaw itself.
Yes, indeed, many new devs overdo things - or lose control during development. My suggestion to any aspiring dev would be to make a short story, maybe a night out with some LI and see how the evening goes, to get a first idea, also at what else goes into writing a game.
As for your last question however I can answer that to you, at least for me and my perspective. I am not interested (here) in watching a story unfold, I want to have some agency when sitting on my computer. Yeah, lying on the couch for flix and chill is all fine, too, but here it just doesn't reach me. And the more options the better - notwithstanding the small detail that too many options will make the game grow out of hand, as we discussed. And it is nice to have some thread holding the bits and pieces together, after all, a total sandbox where I can fly to China and try to corrupt a Buddhist monk would most certainly mean no real characters and stuff, but yeah, kinetic novels (or quasi kinetic novels) just don't do it for me. I've tried them and they failed me. Probably also because I tend to put myself into MC's shoes (unlike people who will play this game as an outside observer like in a book or movie) and I want to earn my sweet little victories, not have them handed to me on a silver platter by just clicking through a story.
And thus we end in "a dev needs to find the sweet spot on the scale". Unfortunately it may be different for almost everybody.
