He did get punched in the face during the the precious fragmentation season 3 episode 17. When he tried to get the ring from Penny while she slept,she punched him in the face.