Well, one, the people who tried to remove him were the organizers, not the ordinary people at the rally. Which could mean they disagreed with him, sure, but could also mean that they thought he was making the rally look bad on the news while still agreeing with what he was saying.