I jumped into Fallout 3 fresh...my first Fallout game. I’m sure there were some references lost on me, but it didn’t seem to make too much of a difference. You really get absorbed into these games by reading all the books and notes you encounter so you’ll be fine.