The toys weren't the problem. It's strongly implied that he spent his entire life in that room (or, at most, the castle) up until Ghetsis sent him out into the world to take on the league. He was also kept apart from human interaction. In the real world, this would be classed as abuse.