Your not alone in that. I was talking to a guy the other day about the same thing and he said what got him the most about it, was when he was trying to catch the girls eye, the guys would often become sexually aggressive towards him or they would turn the entire situation into a joking thing. He was saying it has happened a couple of times and it pisses him off.
Me, Im more of a person that sort of thing rolls off of. I have sharp things to say back to guys like that.
PG