This is correct, the children fought the andals when they first came over to then ally with them to fight the white walkers ( the ultimate big bad) after they had held them off the children and first men became friends. The men then went on to worship the same gods ( trees )who later then became the north men.

The Raft?