“Traffic” is pretty appropriate in my mind. Just the singular word. "Well you are just traffic, you wouldn't understand." Works as easily in plural as it does singular. Plus you have the added benefit of it being an established phrase. Everyone hates traffic, even traffic hates traffic.