I think the White Walkers had to wait for winter to march south. As we can see, being undead does not seem to stop decomposition. So natural refrigeration in the form of winter might be the only way to get an army of the dead south without them rotting to the point of uselessness.