I honestly don't see why refraining from comments about physical appearance is such a big deal to some men.
Personally, I find gendered comments about my appearance very private and intimate so when some dude I don't know says "you look hot" my instant reaction is, 1.eewwww 2. who the hell do you think you are, 3.how do I leave this situation safely. It's one thing to say "you look nice" as this can be said to either gender, to a child, to an old person. But whenever it's sexual, even in the slightest... yeah, nah. If we lived in a world of true equality where women weren't 90% of the victims of violence with 90% of men being perpetrators of said violence, perhaps we could have sexualized remarks anywhere and everywhere and it would be fun. But in the world as it is now?..