Dont really think that it can be counted as Netori, since they dont really have a relationship. Especially not a loving one. That being said, I know people here like Cheating, if it means that there is a woman "rescued" from a scumbag-Husband, even tho this is a little bit of a cliche.
But I would also say that Brians behavior is not that far off when you look at the worldbuilding. I would say that a high member of the founding family treating women as some form of Object or Trophy is not really something which is seen as bad. When it comes to morals, it is, but when it comes to worldbuilding, it is probably more common than MCs approach of a Harem.