Actually, every one of my settlements has a defense of 150 or more and I still have attacks that happen randomly. I believe it stems from how much food your settlements generate.