He says that he would keep Gilly and the baby around to make Sam’s mom happy but I don’t think they would have been safe. I mean, this is the Lord Tarly that threatened to have Sam murdered in a ‘hunting accident’ if he didn’t go to Castle Black and his mom def. wouldn’t have been happy about that. Sam is his natural…