Also just having Asuna would made him more confident and it is scientifically proven that confidence is sexy.
It is absolutely true that one's personality affects their perceived attractiveness, but it's not as simply as having confidence = being hot.
Inoda, at the very least for me, seems like someone more likely than not to develop a superiority complex to hide his insecurities than any real confidence over his acceptance of them.
You can see it pretty often in real life as there is usually one or two in each school, a socially awkward nerd with few or not a single friend, who believe themselves to be better than anyone else for having better grades or their family having money.
And you never see them with bitches throwing themselves at their loner dicks because they believe themselves the smartest brat in the room.
Confidence might be attractive, but arrogance not so much, especially so when it is coupled with a face so ugly that not even his mother would love(literally).