If you're talking about interactive story and scenes, this sounds like a visual novel rather than what most people would regard as a 'game.'
To me, what elevates a visual novel from a long winded comic into something memorable is the emotion it stirs in the player as they go through it. With no emotional effect, you might as well just release a series of images and call it a comic. Getting emotions across though is not easy in words so it'll probably need some carefully designed visual cues as well. Things like facial expressions, poses, etc.
I think we all have different tastes so you're going to get a lot of different answers. Personally, I like a good story. No crazy conflicts that, in one choice, completely reverses everything I've achieved so far. Nice graphics with facial expressions and poses that match the story at that point. I like a bit of a slow burn but not to the point where you wonder if the damn thing will ever be finished. I like a little bit of realism in the story - going from hating someone to adoring them in ten minutes... nope.
So... er... Sorry, after all that, it turns out that pretty much everything is important
Personally, I'd go with story and plot first, then graphics. Whichever way you do it, the graphics will be the most time consuming part and that's the part you don't want to have to keep redoing. Get the story down first then you can create the graphics to fit each scene.
