Well yes, but CHARLOTTE clearly thinks of her as a servant. So if he wants Charlotte to leave his dick alone (say, if he's trying to keep his promise to Lily to stay away from other women on the island), that argument would make perfect sense TO HER, regardless of what the MC actually thinks of Lily.Maybe the MC doesn't actually think of Lily as a simple servant, or even less than any other woman, so that's why he doesn't make that derogatory argument? Just my guess.
I mean, since the very beginning of the game he seems to be highly attached to Lily, like she's a part of the family I would say, and doesn't really tend to treat her badly, unlike how the Hales treat their own servants.