It's nice to have more of them for the extra benefits, not to mention extra income. For a house that big it's hard to imagine just 2 girls doing all the work while also travelling with the master and helping with fights. Someone has to tend the garden, repair the armor, check all the various business deals and so on. The problem is with the time mc spends on doing things. Simply going one step out of the house and back takes like half an hour, a conversation of few sentences can take full hour, a simple bath even more and a foreplay of 2 hours would be boring as hell for the vast majority of people. Mc doesn't have enough time because of unrealistic time requirements for simple actions, not because there are too many girls to take care of.
On a completely unrelated note I believe there is something wrong with the advocate trait if you have multiple girls with this trait. Assuming you don't do any other willpower affecting tasks at first they tend to to average willpower between themselves and once their willpower is equal it all goes down for some reason. It's hard to keep it up at 30+ for the charisma bonus even if you encourage them, keep feminist on Aria and send them all to work that can potentially increase willpower.
On a completely unrelated note I believe there is something wrong with the advocate trait if you have multiple girls with this trait. Assuming you don't do any other willpower affecting tasks at first they tend to to average willpower between themselves and once their willpower is equal it all goes down for some reason. It's hard to keep it up at 30+ for the charisma bonus even if you encourage them, keep feminist on Aria and send them all to work that can potentially increase willpower.