That's not gay, that's voyeuristic. If guys were fucking other guys or the main MC in flashbacks, that would be gay.
It is also most common for people to have different partners and thus be fucking different people over their lifetime (generally not over the same period). If these people are of the opposite sex, that's not gay, that's hetero.
But that's not all, some people fuck different people while being in a relationship, and as long as they don't do it behind their partner's back, it's not cheating any more than fucking people of the opposite sex is gay. Since you seem to like WVM, you certainly know this, as the MC does fuck many women while not cheating on them (last I checked), and these women are not gay.