Yeah, this is one of the secret scenes. Every time you speak to the characters, you build up a Lust stat. (Once per day).
You just have to make sure you don't wander into scenes where the girls climax, as it reduces the lust stat.
It's a bit tough now because there's not that much content, but once the game has 20 main girls, it'll be a situation where you'll be talking to the girls and building up their lust and not reducing it because you're too busy doing whatever girl has an event available.
Oh and I want to make sure you know that the H Scene triggers from working out near your home