Yes and no.
It is a bit deeper than that. Normal men have the innate desire to procreate. Having more than one female, ensures higher chances of heirs/more heirs.
In nature, single life long partners, are an extremely rare thing. Most mammals/animals, fuck as soon as it is heat season. Humans are different in that, since we've got a morality based societies, with various opinions and culture on that. In some culture having multiple wives, is completely normal. For the males, it usually brings both standing, and heirs. For females, a man able to sustain several women and kids, is usually a good bet as far as needs and safety goes.
You've also got to add the fact that not everyone is a good breeding candidate as well. The weakest males, will usually have a much tougher time getting laid. Mankind is once again different in that aspect, since weak men can become powerful if they've got enough money. We're probably the only mammals were the weak can actually strive and reproduce.
But, this also means that having multiple partners, is fairly normal if the weak ones ain't getting as much, since more partners are available, the strongest male will have to breed them as well, otherwise your species would decline pretty quickly, unless like humans, you allow the weakest to reproduce as well.
So no, harems aren't really a "power trip", it's more akin to a natural effect for the species propagation
Sure, it's sexually arousing, but anything arousing you sexually, is in essence, designed to help propagate the species and breed better/more often.
One could make an argument that cuckhold/sharing fetishes, is an adaptation of this as well. The weakest men leaving stronger ones breed their females, because they innhately know it's better for the species as a whole. Maybe a bit far fetched, but when you see how weak most men are today, and how much this trend since to be increasing, it's difficult not to believe it to be at the very least, a possibility.