So far a few things major things can lower the MC's attractiveness in a significant way. Sexual orientation, blood ties (if they don't have an opinion or actively dislikes incest), and ethnicity if you have racism enabled.
Also their opinions on attractiveness can also affect said attractiveness. If you have a character that likes tall, curvy, bald headed, older men and you happen to be a short, fit, younger man with a head of hair, then expect the attractiveness score to be really low.
For me, part of the fun is the narrative that I write in games like these, where it doesn't have an overarching plot; that includes how to I get specific women into bed.
Currently, since it's the first time that I'm playing as a cult leader, the story I write in my head when cult members give birth to my kids is that instead of never being spoken of again, the kids are being communally raised by older cult members that can't or already had kids.
The cult runs a daycare where they take care of kids for free. Ostensibly for everyone but actually all, or most of, the kids that are sent there are mine. LOL. Besides. You can never start too young with the brainwashing.