How important are facial expressions and gestures?

[Mimir's Lab]

Hey guys, I'm making a VN and one of my primary focuses is on not only what is being said by characters in a scene, but how it is being said. I think it's no secret that how dialogue is being said affects the meaning of the words and, given that this is an industry where voice actors are nonexistent and/or expensive, I think the second-best thing to do to convey the different ways something is said is through facial expressions and gestures. I think the face can't hide the truth behind the words being said and so I think it adds a layer of complexity to dialogue that might not be present if that dialogue was said with a straight face all the time. An example of this is if we have a character that says "Fuck you." to the main character. The facial expression he has while saying that can convey a plethora of different emotions and intentions. If he says it with a stern face, we know his intention is malicious, but if he says it with a genuine smile, we can assume he said it in jest.

To get to the point, I think some people read VN's in a way where they consume the dialogue and consequently ignore most images to build up the story in their heads and then use the images to reorientate where they should be. I think this is a symptom of playing visual novels where the images are very limited, requiring the reader to use their imagination to visualize what should be happening versus what is shown on screen. I took the opposite approach in making my VN where the images carry the visualization in the story with the dialogue only appearing where they need to. But I created this poll because now I'm starting to wonder if it's really even worth the effort. Maybe I put too much weight on the importance of visuals in the VN medium.

 

[Ze0s]

Facial expressions and gestures are what give your characters life and personality, especially when working on still images the presentation is everything. If your characters look void of life because the facial expression looks stiff/unnatural people will naturally get turned off by it and probably will hit that CTRL button and skip everything.

If you plan on not having images/CGs, you *have* to have a very deep understanding of how the scene is gonna play out so you can accurately represent it with just enough words for the reader/spectator to not get bored since you don't want to just throw in a wall of text and hope that the reader sits through it lol

Hope this heps you and good luck with your VN!

 

[stevejackhammer]

I took the opposite approach in making my VN where the images carry the visualization in the story with the dialogue only appearing where they need to. But I created this poll because now I'm starting to wonder if it's really even worth the effort. Maybe I put too much weight on the importance of visuals in the VN medium.

Someone who is taking a thoughtful approach of designing the form on the VN before going too far. Well done! For what it's worth, I'm spending a fair amount of time in my design phase dealing exactly with what you're talking about -- expressiveness and the interplay of the visuals and the text. I'm not sure I'm coming to the same balance as you are, but I'm going to invite you to read my blog on the development path I'm on.

I just posted a description of the mechanism I'm using to pull expressive pictures (character 1 is happy, character 2 is upset, and so on) in sync with the dialog as it is revealed on the screen. Check it out and see if there's anything there that might help you as you work out the details of what you want to do.

 

[pastiche]

VNs vary in structure, I find that most Japanese VNs do very little with range of expression and leave most of the descriptive work to dialogue or even CG sets. Tone is usually a binary thing in these types of VNs, when I played Katawa Shoujo the girls would be locked to one of three expressions then say something and the narrator infers the deeper subtext to what they are saying, in other VNs like Tsukihime expressions would convey very little and the narrator would do all the heavy tonal lifting, it takes the novel part of VN very literally.

In some VNs (typically 3d ones) I notice they set scene by frame, Hustle Town is a good example of when a VN uses an image depicting what the characters are doing while the dialogue is happening. Rather than the idle portrait you'll get an image of the characters walking along a creek or smoking a blunt, if the scene drags the context of the dialogue gets lost and it's likely more effort on part of the author to put as much attention to the visuals as they do the dialogue, which is likely why most 3d VNs are typically shorter.

The latter generally forgoes the novel part and the former forgoes the visual part, having a scene conveyed through text but explained through visuals is ideal I'm generally more in line with 3d VNs for this purpose. If I needed to read prose in order to understand something I'm just reading a novel minus character descriptions and even still the narrator in most Japanese VNs will describe the features of a character that you're already looking at.

 

[Adabelitoo]

I, as a dev, consider facial expression and gestures extremely important. I use Koikatsu, which includes a variety of different eyes/brows/mouths but I go as far as playing with the face's bones individualy to move/change things as much as I want. My characters opens and close their mouths while they speak, if they blush, the blush will increasy/decrease gradually, their eyes will always be watching something, never an empty look. The narrator never talks about the characters' feelings, maybe the MC will but only if I want the player to understand that the MC is understanding the girl's feelings, never to inform the player about the girl's feelings.

Now, as a player, I mostly find myself focused on the dialogue and the only way I know that the facial expression changed is thanks to the time that the dialogue disappears for the with Dissolve().

I don't know if I'm so used to other devs not caring about that so I just focus on reading or if I legit don't care that much as a player. I admit that as a dev I'm really picky with my game to the point when it becomes stupidly overwhelming so maybe me caring so much as a dev is just my paranoia and not something I trully consider important.

 

[Nunu312]

I, as a dev, consider facial expression and gestures extremely important. I use Koikatsu, which includes a variety of different eyes/brows/mouths but I go as far as playing with the face's bones individualy to move/change things as much as I want. My characters opens and close their mouths while they speak, if they blush, the blush will increasy/decrease gradually, their eyes will always be watching something, never an empty look. The narrator never talks about the characters' feelings, maybe the MC will but only if I want the player to understand that the MC is understanding the girl's feelings, never to inform the player about the girl's feelings.

Now, as a player, I mostly find myself focused on the dialogue and the only way I know that the facial expression changed is thanks to the time that the dialogue disappears for the with Dissolve().

I don't know if I'm so used to other devs not caring about that so I just focus on reading or if I legit don't care that much as a player. I admit that as a dev I'm really picky with my game to the point when it becomes stupidly overwhelming so maybe me caring so much as a dev is just my paranoia and not something I trully consider important.

I can reasonably easily say you care more than the players. I feel this is one of the things that people assume is a lot more important than it actually is.

I say this from the position of a dev who's game sorely needs more emotion out of my renders. I went old school and have character sprites on top of location images rather than the more common whole scene renders. I straight up don't have enough different poses and expressions, and it's something I want to fix in the very near future, because it pains me greatly.

The feedback? Almost unanimous praise for the graphics and characterisation. One person complained about it (and he's totally right).

 

[fidless]

They're important. I feel that very often DAZ3D characters feel very uncanny and that's because of how poses and expressions are made. Some look good and it depends on effort and skill.
I mark them as essential because very often expressions done wrong feel even gross, especially during sex scenes.

 

[stevejackhammer]

I feel that very often DAZ3D characters feel very uncanny...

The new 8.1 tech for figures uses face bones and weighting to achieve expressions instead of morphs. The result is (allegedly) more natural (subtle) expressions. I have to agree, from what I've seen. Interesting, subtlety is how Daz promotes the 8.1 tech, so it seems like the extreme results you got from morphing was something they noticed as well.

 

[stevejackhammer]

Now, as a player, I mostly find myself focused on the dialogue and the only way I know that the facial expression changed is thanks to the time that the dialogue disappears for the with Dissolve().

Interesting. In my customization of Twine/Sugarcube, the text is persistent but fades partially out as the user reads. That progression also triggers expression changes (if an expression change makes sense, of course), but yeah, how will the reader notice?

Crossfading is too hard (both images need to be on the same point of the page, requiring "absolute" positioning, and image swapping, and this and that and forget about it!), but perhaps a sequence of transitions (one fades out the new one fades in, or zoom out/zoom in, or slide out/slide in) will work.

You've given me something to think about...

 

[imelman]

In most products, I often see characters have like 2-3 emotions at best, which are sex act face, ahego(misspelling is intentional because of dumb cliches associated with it)face, and maybe post-climax relief face, which is strange, since from my view and experience, that's not how humans act, and feels wrong in realistic settings, since humans have very, very different emotions, like being nervous, angry, sadness, lack of emotion if their personality is that, pain, and so on. Having only few makes the work hard to relate, regardless if they are fantastical where magic and stuff exists, including modern era applications in medieval worlds, or a down to earth setting that still manages to scare people off because of same-face dolls having an unbelievable act. I'd clarify that both western and eastern works suffer from this, but its mostly eastern ones from what I seen.

I only saw a few works that do have more than 2-3 emotions, and often they are towards realistic scale, but they are too few and because most people would rather see sex with angels and demons and other unusual beings in a repetitive feeling rather than trying something different and believable. Putting effort into details is important, dismissing them as little things because people are only after sex is fairly ignorant from my perspective.

 

[woody554]

I try to always have something going on in the text, but hint at something else in the image. often it's how or what they're looking at, and sometimes it changes the message completely. the underlying idea is the characters often WANT to think they're portraying themselves in one way, but their body betrays what they really think. I also like it to be unknown by the mc.

but I like it to be subtle and strategic. I absolutely hate the way JABcomics does it for example, where everything that should be hidden is loud in your face. when there are subconscious internal conflicts the character must try hard to sell superficial message, but sometimes there are cracks in the mask and you get a glimpse of what's going on inside.

if actual expressions are used they should be subtle like the acting of anthony hopkins. all of the readymade expressions are just way way wayyy too strong. VN is more like a movie than a play. overacting tends to look hideous.

then again, the nature of the character should also have a big weight on how expressive their face is. a loopy bipolar bimbo might have exaggerated moods while an uptight mature lady should be almost expressionless.

 

[Galm_One]

To me, what the face can show is the one of the many windows to emotions. As they say: "Actions speak louder then words" But don't throw writing out the window because of that phrase!

 

[stevejackhammer]

But don't throw writing out the window because of that phrase!

For me the phrase is "A picture is worth a thousand words" being used as an excuse to not write good words. Too many VNs rely on pictures to replace poor writing or nonsensical plotting. It is a visual novel, not a novelized visual. The visuals are supposed to support the novel, but the writing needs to come first.

Sometimes I see game announcements here that follow the pattern "I'm writing a game that will have an MC and maybe some other particular characters and probably this sort of plot set in this place and time, or maybe a different one", and then 6 detailed Daz models. Wow, how do you get the models done when you aren't even sure what the story is yet?

Like making a movie as a starring vehicle for some hot actor. What's the movie about? Who cares? As long as it starts Hot Actor!

 

[Doorknob22]

Expressions and gestures are important but remember not to go overboard. Some developers dial all their characters emotions to 200%, resulting in what feels like a bunch of hysteric characters swinging madly between rage, ecstasy, despair, and rapture over the most mundane events.

In expressions and gestures, like in many other things in life, less is more.

 

[stevejackhammer]

Expressions and gestures are important but remember not to go overboard.

Another way to look at it is as feedback. A subtle change in the expressions -- enough to be spotted by the eye -- is a way the user know the application is still working. Nothing to do with gaming or immersion or narrative -- just a basic rule that says any piece of software ought not respond to inputs silently. We've all been there. Click something and nothing happens. Is it frozen? Click-click-click-click -- dammit, now I've screwed it up!

I was playing a game where the dialog was an ellipsis ("..."). The idea was the character was thinking. The programmed sequence was three dialog lines with the ellipsis, then the thought or decision happened. Problem was when I clicked, nothing on the screen changed. I did exactly what you'd expect -- clicked rapidly thinking something was wrong and ended up blowing past several lines of subsequent dialog, requiring me to backtrack. Wouldn't have happened if the character image had cycled through different "I'm thinking" expressions. (Just one way to fix it -- could also have made the text different through those three click steps.)

 

[Doorknob22]

No need for different images IMHO. Here is a fresh example from a soon-to-be-released update for my game. By changing the number of dots I convey the passage of time using a single image. Furthermore, if I did include a different image for each "pause section" the result might have been too dynamic for a scene which I intend to be relatively still.

 

[stevejackhammer]

No need for different images IMHO. Here is a fresh example from a soon-to-be-released update for my game. By changing the number of dots I convey the passage of time using a single image. Furthermore, if I did include a different image for each "pause section" the result might have been too dynamic for a scene which I intend to be relatively still.

View attachment 1542035

Quite right! Gives that countdown quality too, letting the user know how many clicks before the story moves forward. I'm just making the point that modern UIs tend to be very dynamic. Click a button, and the button looks like it was pushed. Go to a new page, and the title and navigation slide in instead of just appearing. Stuff like that. If you VN page doesn't have a lot of opportunity for feedback, having expressions shift around is not a bad way to go. Just means a fair amount of artwork.

 

[Diconica]

First, a good story can stand on its own without the images.
An image is the chance for the author to express more visually what they envision.
Facial expression in an image can only convey so much.
When you talk to a person in real life you pick up on all sorts of cues that can never be expressed in a single image or even a video.

Micro expressions. Most people don't notice them but they perceive them subconsciously.
General facial expression people can practice at not showing or even showing false ones.
Micro expressions people can't fake.
Posture, tone of voice, odor, pheromones, touch, vibration and a lot more are stuff we help in communication.
If you take interpersonal communication in college it teaches the value of different aspects like this.

A still image can never show what a video can. You can't see micro expression and you can't even see complete hand gestures posture shifting such as when a person is nervous. You can however write indicating something to that effect. Crime, Detective novels and thrillers are usually good books for seeing this used.

Movies can't give you the sensation of touch, feeling, and more.

Still images and movies both have to convey a message to be perceived. If the underlying development for them is shit the movie will be shit even a painters works will turn out crap if they can't figure out what they want to convey.

So no matter what the first priority should be the story the underlying written version of what you want to convey.
Great novelist didn't get to their level worrying about images.
Just look at the image as an extension.