Yes he should have recused himself and the argument about the SCOTUS is apple to oranges. The lawyers arguing the cases are not the defendants.