It does not freak me out as such but I do try to be aware of anyone(s) following me, either walking or in the car. And in the right, or wrong, environment, it can be cause for concern or, should I say, heightened awareness and readiness. As others, I use devices and the environment to determine how close someone following me is, from footfalls, to keeping an eye on the reflection in house/car/store windows. At night and depending on their placement and angle, street lighting can also be good for gauging if someone is closing in on you by the closeness of their shadow without you having to fervently rubber-neck all the time.
Now and then when walking I have simply gone to the side of the foot path and pulled out my phone for a fake call to let such and such go by. I am more concerned about someone I have cause for concern over following me and so learning my home or work address rather than just being followed in itself. You can pretend to txt or call but a fake call is better as it is quite natural to look about while on a call and you don't need to look at the phone and be distracted while fake-txting. This is of course for the situation where you are more concerned at simply having someone on your back so that the odds are in their favour rather than that person catching up to you (ie if you are a female or smaller chap and there is some lumbering Godzilla dude on your heels that you don't necessarily want to have to confront).
Crowds as such do not bother me at all, such as at a sports event or concert or even a particularly busy street. It's always good of course to monitor the pulse and vibe of a crowd so you know if things are going to kick off. However, I may be naïve but I think the chance of having some random stranger slip a stiletto into your back while in a crowd is pretty damn rare and much rarer than you being jumped in some back alley or dark street while on your own. I'm more concerned then about a bit of opportunistic pick-pocketing and generally transfer my wallet to my front pocket or into my inside jacket pocket.
Stay safe out there.