Our MC is already someone outside of the norm, as he is fated to be the husband of a Lilim. Hell, generally MCs are outside of the norm. Since Lilims (like their mother) can have reality-manipulation and warping on the level of a god, there's almost nothing that is outside the realm of possibility in this setting.
Outside of that, consider the fact that anyone has the potential to break out of what's considered normal, as incubi do break through the limits of humans, and anyone has been purported to be able to train and become immensely powerful over time, it's just that such ambition is rare, whereas potential isn't.
Also, there are games with harems of over 50 members, so it's not a nightmare, unless a lot of focus is put on individual wives. Which is fine, if that is what you want. I'm not certain exactly what kenko intends here, and honestly, I think I would be fine either way (small harem with lots of scenes for each one vs colossal harem with few scenes for each).