Maybe because Negan reminds us a little too much of a certain person who was recently elected? I like to work out my anxiety by getting scared of imaginary zombies, not have it amplified by getting scared of an imaginary dictatorial narcissist.