You have to keep context in mind. Robert's bastards only became important after it became somewhat common knowledge that Joffrey and his siblings weren't true Baratheons. Something Ned could not have known would be the case 15 years before it happened.