That's true. Your intelligence is independent of the degree you hold. I'm sure there are intelligent people without advanced degrees and some people with advanced degrees that make you wonder how they got them. If the job you want doesn't require a high level degree, there's no real reason for you to try to get one, is there? There's nothing wrong with that. Different people take different paths in life and spending a long time in formal schooling isn't best for everyone. So don't be so concerned about it.
BTW, since I'm writing, I might add a little comment, something people seem to miss/forget sometimes. Frequently, the higher the degree, the more specialized it is. High school basically finishes out your basic education requirements, knowing something about everything. When you move to college and a bachelors degree, you're doing almost the same thing, but picking one field to focus on. So your education is still pretty well-rounded. Moving on to grad school means more specialization. At the masters level, you're studying only one field and at the PhD level, you're studying only one small area of that field. So someone with a PhD still has a bachelors level knowledge in almost everything outside his/her main field of study. Sometimes I think people put PhDs higher than they really are. Yes, it takes a heck of a lot of work to reach that level, but it's a very specialized degree; the general knowledge of PhDs is still normally at a BS/BA level, just like many other people. FWIW.