I think even his feelings on Jimmy are nuanced. He's totally right in his view of all of Jimmy's bad traits. But he also overlooks the good, and he clearly responds to those bad traits in really snakey ways.

To be fair he yelled at Ernesto because he was having a horrible anxiety attack

Holy hell, I don't even have words. First time I've ever cried from any media. It was just a few seconds of crying, but still. God damn that ending was perfect. I was so excited for Chuck having the pride to ditch the space blanket, but as soon as he started getting off-kilter I realized it'd go bad… but I didn't…