The problem is that the rate of space expansion at great distances (e.g., >4200 megaparsecs) exceeds the speed of light. So at a certain distance, space is pulling things apart too fast for the light or gravity to make it back, influence matter, bring them together, etc.