I've clicked on every person and item in the plague town. Still can't find the milk and apparently the milk from my harem girls isn't good enough. I assume it has to be in the town: "all the items you need".
Someone's probably mentioned this already too. But when you tell the harem trainer to train the mother or daughter while you're north of the trainer, the girls will appear on either side of the trainer and cut off the party from moving. They essentially form a wall that locks you in. Seems to go away by itself once training is complete though.
Edit: I had already randomly destroyed the scaffolding while clicking on everything in the town. Went back here to see if anyone had any instructions and found instructions to click on the roof, but that didn't work the first go-around either. Finally did the second try.
Feedback: There needs to be some sort of hint here to even bring you to the scaffolding as well as some direction of what to do afterwards. Otherwise, this portion really is just random pixel hunting. For game design needs, this portion could really use a bit of tlc. In a different game engine where suspicious-looking items in the scenery can call attention to themselves based on the way they're displayed, hints might not be as needed. But in RPGM, it's all just a blotch of pixels in these scenarios.
Edit 2: Just saw the dev's post that this release was unfinished and leaked unexpectedly, so I'm sure he's working on the scaffolding thing as well as other behavior like the dialogue in the town.