I think that some of the problem is that the characteristics labeled as asexual by people who identify as asexual are so broad as to be basically useless. It also leads to some people framing it as “...unlike you...” when what they’re describing could absolutely apply to someone who doesn’t identify as asexual.