I’m not very informed on the papacy but I heard on Reddit that he had wrist surgery on that hand and it still bothers him sometimes, so when the lady grabs his bum hand, he’s not just annoyed, he’s in physical pain. You can see the pain flash for a moment in his expression. I think THAT’s why he reacted so quickly…