Yes. That’s the risk. Early in my transition I could choose to use the women’s restroom, and get screamed at for being a boy in the restroom, or going in the men’s and getting beaten up. Hell, I worked at a law firm, and a coworker came at me in the men’s room. A freaking law firm.