The heart level indicates how much they care about you, but it's not the only determining factor in whether or not they'll sleep with you. A girl who isn't horny won't be in the mood for sex even at 5 hearts (although you don't want their affection lowering anyway, since they tend to stop putting out no matter what). The Book of Girls will also let you know if they're horny, so that helps.
Aside from the Special Pastries, you can also Awe women (under Skills in the main menu) once you complete the vampire questline and bring Aodabi someone to drain for the Blood Potion. Awing someone both makes them horny and improves affection (I can't recall if the Pastries improve affection). IIRC it's the lowest-level power available, meaning that one Blood Potion will give you 3 days of being able to improve relations without needing to woo them of give out Pastries.
Then there's the mod: one of its functions is to stop affection from decreasing. Good for when you just can't be bothered anymore and just want to sleep through everyone's pregnancies until the next mission starts.
