It's the story that drives the game. It's often what captivates and titillates the player or drives them away. Be it deep with expansive branches, or slapped-on superficial ones, they matter. Just look a the tags for a game, for instance. Some players seek certain situations while others will go out of their way to avoid certain topics. Many, many game threads on F95 often have heated debates of the merits of this or that in game; why certain routes, plotlines, or characters are desired and be expanded, or should be culled and be axed entirely, and everything in between, regardless of the visuals.
Yes, it's true that there are many players that just looks for the eye-candy that they like, or use it as one of their first criteria in what constitutes a good game for them. I won't deny that and it's a perfectly reasonable opinion. However, I believe that there are more players out there that do care about the story to some degree than not. Otherwise, why wouldn't most of us just go to a gallery or a porn site to view all the animations and CGs in a game? It'd be a much easier and a quicker way to view them after all. That's because plot matters; and, hopefully, it's something that particular player enjoys. (In fact, many of my favorite games are solely text-based outside the UI; whereas I've completely dropped other games despite some rather fantastic visuals due to the poor writing or plot.)
The visuals, if any, should be there to enhance the experience; not mar it. It's only a facet to a game. You wouldn't want a happy-go-lucky type of game filled with thematically dark and gory visuals nor would hyper stylized visuals be appropriate in a hard-nosed, super serious type of game. It shouldn't be a disjointed exercise. The story and the visuals should compliment one another. Besides, isn't it much more interesting to know who the character(s) are in a CG and what's going on at that moment versus just looking at a picture without any reference whatsoever? Knowing the context often elevates the visuals to another level than otherwise. For most of the players on here, I believe, and regardless of how much of the text of a game is read, the story matters.