My suspicion is that either a few or all of the girls are imaginary, which is why their personalities are dependent on the MC's mood. Like, if he's relatively well, then everyone he talks to has the patience of a therapist, but if he's not so well, then people ignore him. This would also mean that you're either not killing off a real person, or if you happen to choose a real person, then the imaginary monster can't actually do anything to them.
Another possibilty is that magic is real and there's some spell shit going with butterflies and the book and the MC can control the minds of the women in some way, which is why they suddenly think he's hot.