“My answer is always a polite “if you have a warrant or probable cause then you don’t need my permission. Do you have either?” When they say “no”, then I follow up with “I’m not interested in giving up my rights so if you need my permission, the answer is ‘no’”.