Yeah, the thing with blackface was it wasn't merely used to let white actors play black characters; it was a widespread and highly codified method of mocking black people and invoking racist stereotypes. That's why it carries so much negative baggage.