You adapt over time. As you do, you gain confidance. As you do, people feel less like they can get away with this stupidity. You'll probably always get some but there's less and less over time.
Doesn't mean it feels any better when it does happen, but the less it does the easier it is to brush off and the faster it is to forget.