Is basically down to how people wants to perceive or exaggerate things, really, and/or how they define the 'Harem' genre in anime. Are there several girls who like Kirito? Yes, but all of them respect his and Asuna's relationship even if for a couple of them it broke their hearts when they had to admit it, and despite liking him they still mostly treat him as just a friend aside from some 'slip-ups' here and there (Which to be fair are almost nonexistent in Canon and only really happen often in the SAO Gameverse AU of the console games), while Kirito firmly treats them as friends/family.
Something like TLR can be more easily called a Harem because the MC is for all intents and purposes not in a relationship and he HAS shown some degree of mutual attraction to several of the girls that are into him, even if nothing ever comes out of it for real.
'True' Harems are super rare, but they do indeed exist, they're just so niche compared to the second (The most cliche) situation that all of them usually get labeled the same (For example, DxD is a true harem, Issei is not only actively aiming for one, he manages to get all the girls and legit marry and have children with them. Testament of Sister New Devil is another one, though that one has their novels become literal Hentai by the end of it as the MC gets to fuck and impregnate his whole harem onscreen by the end of it).