It's a complicated mixture of age-old racism, a slight understanding of white privilege and a fear of payback from those oppressed by white people over hundreds of years. Which gets all super fetishized and packaged into all those fears you hear your bf talk about.