Quick Google search came up with a BBC article from 2017 that quotes a NYT article saying that there are about 3000 members total of the two largest Nazi parties in the US. Probably a bit larger now but not really in proportion to the attention they get.