You just put the fear of being sexually harassed for politely saying no on the same level as being reluctant to be politely declined. No one likes to be rejected (and you wonder why women smile and turn away -thank us for making it easy on you) but at least you don't have to worry about violence.