Just don’t do it. The problem isn’t the use of a specific color, it’s the comparison between non-white people (and it’s almost always non-white people) and non-existent people with strangely-colored skin.