I can see this being a problem for the transgendered. And by those, I mean all transgendered. I think a lot of people don't understand the difference between TV and TG. A transvestite is someone who enjoys dressing in the clothes of the opposite sex, but does not necessarily want to become that sex. A Trans-gendered person can be either a person born with a definite sex who wishes to become a member of the other sex, or they can be a person who has been born with sex traits of both genders and has had - or desires to have - sexual reassignment surgery to become completely one of the two genders. These latter are often known as hermaphrodites, and are more common that people suppose.
The laws of various states have allowed people to take on the legal identity of another gender either after or during gender reassignment; either before or after the actual surgery, so to speak.
So a person could be legally permitted to use the bathroom of a given gender while still possessing some of the traits of the other gender. In fact, they'd be prohibited from using the restroom of their previous gender.
And of course, there are people who appear to be one or another and are not. I have certainly met some women who possessed certain man-like characteristics and were neither males nor wished to be one. What rights do they have?
I realize that all niceties to the contrary, there are still some neighborhoods where people dare not be different.