For me it was the accent. Harris was Irish but didn’t play Dumbledore with an Irish accent. Gambon, although Irish, did not himself have an Irish accent after moving to England when he was young, but for some reason decided to put one on. And at least in his first Potter film, it was not great. Still a fantastic actor…