liability... the insurance costs of opening your track to a bunch of kids without helmets or, otherwise, safety-confirmed vehicles are enough to prevent this from being a thing.
That, and the fact that the tracks that I know of in SoCal are a good 100 miles away (Buttonwillow, Willow Springs, etc.)