Also, it's one of those great Moffat "Jesus Christ, it's a fucking TIME machine," moments. Of course he can just leave them behind and revisit them whenever he feels like it - even if it's subjectively three minutes for them and three decades for him - as long as they don't get angeled.