The Doctor said they found Clara through his memories after he was attacked. Then from Clara they spread out to other Earth inhabitants and they became "collateral damage".

All the others are probably dead, there was no-one around to tell them it was still a dream. Although you could also argue that the others might not have been subjected to the same number of layers of dreams as Clara and the Doctor had.

Why do I see nobody asking where the Doctor was when he was face-hugged? Did he really find Gallifrey with the Master's coordinates? The place was all fiery and exploding, just like we last saw it before it was frozen in a time pocket.