The “slippery slope” argument is an easy one to make which is why it gets trotted out so often, but it’s also a lazy argument because it can get extended off in any direction to infinity. You have to draw a line somewhere. Punching (neo-)Nazis is an acceptable line to stand behind.