The argument is that, according to the Histories of Westeros, Rhaegar’s father—who was quite fixated on Tywin’s wife—went and had a little extended, uninvited (but he’s the king, so whaddya gonna do?) visit with the Lannisters sometime before Robert’s Rebellion.