OK, after I watched the third time trying to squeeze some more information I noticed the flashback ISN'T Jake telling the tale.
From Finn's point of view Jake simply say "It was a day like any other. And then they never told me."
This tale is dead and buried with their parents.