Women shouldn’t do that either, don’t you think?
Much as people may look regardless, I suspect the men who profess to be so afraid of being called out for looking are really being called out for doing much more than looking.
Just try walking around the world with large breasts. People think you’ve given them permanent permission.