Sidepiece? (dismissive but unisex)

Too much of a proper common title for me.

Please stop referring a woman who sleeps with a man who is married as a “mistress.” There isn’t an equivalency for men; it’s an offensive and sexist word.