He dismissed him by signaling to him to leave. Going to have a conversation only escalates things and (more importantly) delays the game. The manager isn't owed an explanation during the game and is merely given one as a courtesy that doesn't always have to be extended.

Cards of any color are never shown to team officials. They are simply asked to leave usually after the “Ask, Tell, Dismiss” guidelines have been followed. The match official clearly gestured to the manager to leave and that is enough. The manager isn’t owed an explanation during the game as the game must be allowed to…