This is what always confuses me about comics; why can't this industry adapt to the reality of what the market wants?

The other underlying issue here is that I do not think that readers are not interested in diversity but perhaps they are more willing to see new books with new characters than they are to seeing a new individual(?) portraying a character that they love.

Exactly!

Every time I get hooked on a title I have to sacrifice following a story that I am enjoying and emotionally invested in so that Marvel can launch some kind of inane crossover which ends up being a giant disruption.