It goes back even earlier than that. I still vividly recall how disinterested he was in Gohan when they reunited on Namek. But yeah, he’s a pretty lousy guy when it comes to interpersonal relationships. Hell, I think Goten might receive more face time and parenting from Vegeta than Goku!