Well, short answer... It's complicated. Long answer... lots of reading required
There are two basic options at this point.
a) remove the "girl favor" completely from the game (which I don't really want to do)
b) put the "girl favor" to some use
Let's say we try option b) for now.
So how can it work?
You either have to be asked if you want to spend the favor (or not), which interrupts the game flow and can feel strange like seen in the Desire scene. Or the game decides on it's own to continue if you have enough favor (so kind of auto use it if you have it). If you don't have it, the girl will react in a way to let you know that not having enough favor "stopped" the scene.
The "game flow" problem can possibly made less severe if presented in a different way. More like give the choice first to ask for a favor. To stay with the Desire example, maybe the mc could ask her to "jerk him off" or similar. Maybe like that:
"ask to get a hand treatment (might require favor)"
"continue talking"
If love or affection or lust or whatever of girl is high enough, she will do it without the need of favor.
If it's not high enough, but you have favor, it will be auto used.
If you don't have favor, she will replay accordingly.
In general, you can always argue that once she loves the mc or affection is high enough, she 'd do anything for the mc anyway without the need of "favor", which would reduce the favor use to start of game or maybe mid game. After that it would become obsolete, which would go into the direction of removing the favor completely.