"Dating someone who's being unfaithful isn't 'wrong'" - You're doing something that will cause harm to another person (and possibly children, since this story never mentioned whether or not the married guy had kids) who has never done anything to harm you. Potentially ruining lives.