I just said it in another comment, but I have a feeling he thinks (as a lot of older men do) that men don’t really have the right to use the word “feminist”. He’s always stood up for women’s rights so I don’t think he meant to have it sound insulting at all.