It's funny, because I only made the connection once you made that comment. It didn't actually click during the show.

Jack knew Hannibal was at the Palazzo because of the brochure/pamphlet that Pazzi picked up earlier and brought home. You can see him looking it over when he's with Pazzi's wife while she was worrying over his absence.