That's right!In chapter 4 if you stay in your room and skip time, eventually you will arrive at the time the next main story line event triggers. It will then tell you where you need to go. Other than chapter 1, I don't think you have to complete any of the girls events to continue to the next chapter, just do the story line. Chapter 2 you just have to get the house ready for your aunt. 3 and 4 I think you just have to do the story line events. Optimus please correct me if I am wrong.
In Chapter 1, you must complete all the events linked to the girls in the manor.
From Chapter 2 onwards, it's no longer mandatory to complete all scenes. In the afternoon, you'll see a special icon on the map to continue the story.
From Chapter 3 onwards, you'll see hints appearing at certain times of the day, so all you have to do is follow them.
Example :
