Grew slowly at first by just sorting by damage and putting the high damages together, keeping my infamy low and just occasionally getting some extra women. Then you send some slaves to make orc babies, get orcs, then you get onis and it's easy-peasy from there. Got to day 40 with maximum infamy for the last 10 days, what a breeze. Once you get two sisters and put them on the pillory they'll keep your boys alive too; saves on having to replenish them.
Oh and if you spam drag monsters onto a stockade it will go above the cap lol; having 20 shortstack archers helps quite a bit.