Then you want to learn how to fight effectively. So whatever school you join, you'll need to make sure that they spar often and teach good techniques. This will cover the physical elements of self-defense. As you are training to be physically capable to fight at the level you are working towards, then find some good self-defense books and take a valid self-defense class that has good non-physical self-defense methods.
A lot of self-defense actually happens before the the confrontation. There are 2 reasons people get harassed. 1. They either look like an easy target or 2. They look like the "big dog in the room." Jealousy is a powerful thing and is often at the root of some of the harassment that people have to deal with. Once you know what box you are in, you can do some non-physical things that will help prevent things from getting to the physical fight stage. During my periods of harassment, I was mostly in the #2 box, not because I was the "big dog in the room." but because I don't bend to the will others easily. I'm the guy that is happy to break off from a group to eat where I want to eat, even if it means that I'm going to eat alone. Cliques were never were my thing and I didn't mind being independent of thought and action. It wasn't that I was trying to show that I'm better than others, it's just that I'm naturally comfortable being with myself and not always having to be part of a group.
I'm gentle by nature and nice by nature and many people take that as a weakness so from time to time, that put me into box #1. I have had women tell me that "I'm too nice." and it make it seem like I can't protect my girlfriend. That nonsense changed when I was in my 30's. People have a lot of misconceptions about life and what it is supposed to be like.
Now that I'm in my 40's I rarely get harassed and when I do, I can diffuse the anger really quick. I definitely had a lot practice in doing that. I train to fight but to be honest at this point, I think it's going to be very unlikely that I'll be in a real street fight again, but I keep up my training just in case I need to fight or run from one. I think the biggest change for me is when I had my own family. I'm a very protective person by nature and I think my family brings that in a way that it shows on the outside for the world to see. There are some things that I don't do anymore because I'm afraid it will put my family in danger. If it was just me, then I would probably still behave in that way, knowing that if things got to a physical fight that I could do well with that, provided that there are no weapons involved meaning that my attacker had one and I didn't.
Sometimes the crap we get in life is the result of how we travel through / interact with various environments. The rest is just A-Holes who have nothing better to do then pee on another's parade.
Wow man, thank you so much. There's so much good advice here. I'm stunned by all the positive answers and people actually sharing their life experiences in order to help me deal with my issues. If only a portion of this kind of acceptance translates into a kickboxing gym, I'll be glad to never miss a training session.