I always thought that one of George RR's failings was that he wrote it as if people would just dismiss the incest rumor once it started. People love a scandalous rumor, especially when it involves pretty people. People still like to talk about the most likely completely fabricated accusations against Anne Boleyn.