I'm not certain if we even know that they weren't together at the time. Obviously from her point of view they weren't. But it's hard to say for sure whether that was communicated to him or not. After all why else would he have a key to the room they were doing the nasty in?