You have to wait until you get an invitation to join the girls for dinner, but you have a few things you need to do first before you can get to that point.
Do Rebecca's hint first (at 07:00 the following morning). Take Madeline to her board meeting, then go to your room, change the time to 23:00 and go to bed. You'll get a message from the blackmailer. Take Annabelle to her art class on Friday night, then go to your room. You'll get another message from the blackmailer. Go to the basement Sunday morning at 07:00 and help Madeline with her workout. Then change the time to 17:00. Rebecca will give you an invite to join the girls for dinner. Then follow Olivia's hints after you join the girls for dinner.