Flag On The Moon, I think that you're overstating the rigour involved in common usage of this expression. I agree with what I think Andrew was wanting to say, which is that in common usage the saying means "I can't be bothered going to the effort of working out if my "rule" needs adjusting or not, so I'll just treat…

By the way, I should mention Andrew that I actually agree with much of what you're trying to say about the scientific method.

Gravity is a fundamental force. Newtown developed a law to try to describe it based on empirical observation. That law works pretty well at a macro level but Einstein showed that the very small scale was an exception.

While some of the argument is pedantic I totally agree that the saying is generally used by sloppy thinkers to avoid considering why the exception doesn't fit with the heuristic that they think of as a rule.

Gravity isn't a rule.

You're possibly technically correct Andrew Coleman that a rule can't have any exceptions but you must agree that "that's the exception that proves the heuristic" doesn't have the same ring to it!