How difficult is it to just tell the guy, can you please not fly close to me, can you please not video anything Im doing. Not everyone with a drone is trying to bother others, just having some fun.

So you assume the drone guy was trying to be an asshole? He was just flying for fun & was not trying to harm anyone or invade any privacy. People are taking pics with their camera all day long, nobody says anything, the moment its on a drone, all hell breaks loose. You are the Fucking asshole!