kai, what is the law in your area? Look it up on your state attorney generals website (assuming you live in US), don't just ask some friends "who know", or LEO's. In my experience, 90% of the advice I've gotten from people, including police officers, was NOT the law. Find out what the laws are, if it's different from the common knowledge spouted off by everyone you know, print them out with the code number for your state and carry them with you. If you run into a police officer ignorant of the law, show them what the laws say. If you are legally allowed to carry a knife, then you're legally allowed to carry one. If you're not, or they have a set of conditions on what you can carry and how, follow those conditions and the police will have no reason to bother you. If knives aren't legal in your area, look at what legal self defense options you have, (pepper spray, taser, firearm, etc).
Like them or not, the police are there to protect the public and enforce the laws of society. That 60 year old man growing medicinal cannibus, I assume he didn't have a license to do so legally? That vulnerable woman carrying the knife, was the knife legally carried or fit into a set of legal parameters (size, length, etc)? Police don't make up the laws they have to enforce. I know some officers that hate the laws they have to enforce and think them horrible, but if they don't do their job, they get fired and can face their own legal consequences. If the laws are not to your liking, write your local representatives and call them to try to get them changed. You'd be surprised how many of them will actually talk to the public. I've actually spoken to the governor of my state on a bill that was coming up for vote a few years back when I called his office. Imagine my surprise when the governor bothered to speak to a nobody like me (blue collar job, no political connections or ties) when I bothered to call. Get enough like minded people to bother them with well reasoned arguments and things happen. If they don't make the changes, vote them out of office next term.
Don't hate the police for having to enforce the things the politicians make them, and don't fear them for the same things. Work towards getting them changed.