Flying Crane and Harlan, you've both brought up terrific points on this particular subject, and I don't disagree with you. Trust certainly plays a role, Flying Crane. Building trust is actually it's own skill set. Harlan, I also agree with you that a person's own expertise will be a factor in how well that person can judge someone else's expertise.
I want to shift the perspective here. I'm trying to look at this through an outside lens. In other words, whether you can accurately gauge a person's expertise, that person either is or isn't an expert in a particular field. A martial arts instructor is either a bone fide expert in that style of MA, or he/she is not. A surgeon either is or is not an expert in that field.
A complete novice might not be able to tell which is which, but that does not affect the instructor's credentials.
I guess what I'm trying to get at here is that I see your points, and they're good ones. But I'm still wondering what you think about the actual development of expertise as an individual. You are trying to become an expert in something. What are the ingredients? What are you actually learning? What does an expert of something look like to you? What are the characteristics of an expert?
as I think about this, I'm not certain that there is a single, solid answer to this. It might not be something that can be clearly and objectively measured up to "thus"...
As Rframe states, an expert is one who has knowledge and the ability to apply and correlate the material. But how is that measured, and at what level of this ability is enough to be an expert, or a master, or whatever you want to call it? Development of skill is a progression and it's difficult to measure and difficult to assign a solid value to. It's very nebulous. that's why I keep coming back to, everyone needs to decide who they trust to give them the info and the training. 'Cause I'm not sure how else to say it, I can't give definite parameters. As soon as I try to nail it down, I suspect someone will point out an example that doesn't fit my definition.
And keep in mind, the "best" in the world (if that could even be measured) is still vulnerable to a suckerpunch or being blindsided, or bad luck, or a bad day where nothing goes right, and could still lose to a Nobody on the street. It can happen at any time. So what does it really mean to be an expert? It does not mean invincibility. The Best who loses to Nobody is still worth training under, because he's got something worth learning. But the fact that he lost to Nobody, well now his reputation is tarnished and people suddenly doubt his ability, maybe he's really just a charlatan.
I'm kinda thinking out loud here, just rambling in my thoughts I guess.
I made the example earlier of someone with crude technique who can still be very effective with it because he's big and strong and athletic and aggressive. OK, does that make him an expert, does it make him knowledgeable? He's certainly effective, but can he actually teach what he does to someone else, and will that person be as effective as him if he lacks the same size, strength, athleticism and aggressiveness? What does he really have to teach, in this example? He may have found something that works for him, is good enough for him, but that he cannot pass along to anyone else if he were to try. At least not to anyone who does not match his natural attributes. This is someone who is not worth studying under because he has very little to actually teach, even tho he has the ability to clean out an alley full of ruffians.
What is perceived as important to gain a level of expertise is something that I think changes with one's experience. A novice sees only the color of a belt, and thinks that is a measurement he can count on. The higher the rank, the better the instructor. We all know that's not a trustworthy measurement.
I've spent years training in several different martial systems, all with a very different approach to how the curriculum is organized and trained. What I thought was a good system when I was younger and less experienced, is stuff that I now feel is not such a good system, or at least not a good match for me personally.
I guess for me, it comes down to a couple of things. First, you need to study and train a system that makes sense to you and is a good match for your personality and how you mentally order things in your cognition. You need to train your techniques to be strong, you need to work them in a way that helps you be able to use them on an opponent, but how you go about doing that is subject to much debate. Is competition necessary, or not? What KIND of competition? Are interactive drills enough? How about drills that are scripted, vs. drills that are more spontaneous? Perhaps they are all a part of the big picture, they all have a place in the progression of training and none of them alone are THE answer.
I know my sifu's training history, I know who his teachers were, I know that when he was young in Hong Kong in the 1950s that he got in fights and used his stuff for real. I understand our system enough to know that it's a good match for me mentally and the system makes sense to me. When my sifu teaches, what he says, what he shows us, makes sense to me. And I've got enough years in (close to 30 now), with experience in enough variety, that in my opinion, I can distinguish nonsense from quality. What I see in my sifu is quality. not because he gets up and dazzles me with his mad skillz. But rather because he can explain in a way that makes clear sense to me. And when he demonstrates his point or examples he wants us to see, he gets huge results from very simple things.
But again, it's just my judgement, I've made a decision that I can trust his teaching.
So I dunno how else to answer your question. It's a good question but perhaps not one with a clear answer.