As a Canadian who travels to the US 6-8x a year, I /never/ carry my passport with me, except when crossing the border. I was once refused alcohol because I didn’t have it on me (they didn’t accept my license plus a half dozen other pieces of ID). I thought - and still think - that was pretty stupid.