While you're totally right in saying that fun factor really is the most important thing about a game, the problem is that not everyone thinks the same things are fun. Which is why having a GOTY is pointless there's going to be so many different GOTY votes that it makes everything moot.