I think even more than that, it was the fact that in the books anything that comes to life in the cupboard is a real person being pulled from some point in history. If Darth Vader and Duke can come to life, does that mean they're real? It just goes a step too far outside the reality the story creates for itself.