It’s actually not Bran’s fault: That’s the thing. He just happened to be there.
What I got from the scene was in that exact moment young Hodor was experiencing everything from that moment and time... and old Hodor knew it. That’s why Hodor was freaking out well before things started happening. He knew it was how he…