I was under the impression that the memory wipe only worked partially. As the Doctor explained to diner-Clara, he remembered that Clara existed and what adventures they had together. But he can't remember what she looks like or sounded like or how he got back to Earth.