I've heard it as "Joffrey" before I was ever aware of GoT, and before many others as well I bet. Geoffrey used to be pronounced the same way as George (both old english names), for some reason the "jeff" pronunciation got more popular.

I think Wilfred referred to the kid's name as being spelled with a G, as in Geoffrey.