I don’t think that’s necessarily true. Those “average” men often have a deep belief in their own undesirability and they don’t try.
Perfect example: the guy in my literature class who’s trying to get my attention has a partner and is obviously a jerk, and is only a little bit cuter than the other guys in class, but I do feel the chemistry because he’s trying so hard. Plus I’m flattered by how hard he’s trying.
Meanwhile there are several other guys I’d be interested in dating if they asked me out but I have no idea if they even like me a little. They don’t meet my eyes or pay any attention to me at all. I have no idea if they’re married or gay or what.
Guys often lose opportunities because they don’t communicate their availability or preferences.