Besides the fact that it's perfectly fine to be intolerant of intolerance and that such behavior is, in fact, what all societies should strive for seeing as exclusion based on something like skin color is antithetical to a society that can survive, thrive and grow, no, I suppose there's nothing wrong with your logic.