And even then the legalities are sticky...
In Colorado, at least, we have a law about 'mutual combat' law, that basically says if you both agree to the fight, it's not an assault. At most, you can be charged with misdemeanor disorderly conduct. And according to the family cop, even that is unlikely unless you do it in a place that disrupts other people. If you just go off in an empty field and wallop on each other, nobody cares.
And it's easy to agree. If you say you're going to smack me upside the head, and I say "Go ahead and try, lunchmeat!" I can be considered to have agreed to the fight.
At least, that is my understanding, as a non-LEO, based on my experiences and conversations with cops.