What Sam said to Olly was "Sometimes a man has to make hard choices. Choices that might look wrong to others, but you know are right, in the long run," right after Olly worried about the Wildlings slitting the Night Watches throats, and remembering that the new leader of the Wildlings led the butchery against his…