Anecdotal vs. empirical

[Gerry Seymour]

Your second paragraph is exactly why there's the issue. If you have one definition and I have another, how do we choose which definition to use? Or are we both trying to refer to the other persons definition while also talking about our own? It just opens up the door for confusion.

Also, this thread shows that it's not malicious. Your take, my take and Steve's take on SD were all different, which played a part in your debate. I assumed I had been clear with mine (reading over it again it's very clear I was not), and I doubt either you or Steve have the intent of "strawman arguments".

As with any conversation, we have to choose something. We can't discuss "engineering" without agreeing what falls within that area (do industrial engineering, software engineering, etc.?).

Generally, one of two approaches is pretty successful, unless the definition of the term is the actual point of discussion. If you posted first, I usually try to work with your definition, noting that mine is different (to avoid confusion in other threads/discussions). Or, you and I take hold of whichever best facilitates the discussion (most expansive or most restrictive). That requires some cooperation, though. Some folks will simply not cooperate. There's plenty @Steve and I disagree on, but we usually manage to agree on the usage of terms in any given discussion (again, unless the usage of terms is what's under debate).

Of course, if I want to be difficult, I can simply keep using my definition to show flaws in your statements (which there will be, since your statements were made with a different definition). So, if you are talking about Jeeps (uppercase, the brand) and their properties, I could keep insinuating statements about other vehicles commonly included in the "jeep" (lowercase, casual usage, not the brand) category. Every error I point out would be factually correct, but inappropriate to your actual statements. This is one of the most infuriating types of strawman arguments (and something I see on MT from time to time).

 

[Gerry Seymour]

Time is short, but I have two quick points, and a question.

First, defining Self Defense is often opportunistic, and shaped by our experiences in the past, and the position we're promoting. It's backwards.

Second, most people cannot even pick a single definition and stick with it, me included. Context matters. I tend to think of self defense as being things you can do to reduce real risk. But self defense is actually not any action at all. It's a justification for breaking a law... a legal defense. Self defense isn't what you do in the moment. It's what you tell the judge after the fact.

The question is this: What would you think if you saw someone dismiss empirical evidence in favor of anecdotal evidence?

If the empirical evidence is sound, it should carry more weight than anecdotal. If the empirical evidence is flawed or insufficient, the anecdotal evidence (if there's a significant amount of it) may be of similar weight.

At the very least, the anecdotal evidence should be used to challenge conclusions from empirical evidence, to make sure we aren't either shopping for statistics, or ignoring less-likely-but-not-improbable outcomes.

 

[Gerry Seymour]

I think it provides a very good example of how self defense can be approached empirically. And in fact, I think it suggests that attempts to draw conclusions across a larger population and range of possible attacks are exactly the problem. it's like saying chemotherapy works for cancer, and then concluding that everyone who goes to the doctor regardless of malady should be treated with chemotherapy.

I'd put even a finer point on that, Steve. It's like saying chemotherapy works for lymphoma, and concluding that everyone who has cancer should be treated with chemotherapy.

We'd need to study subsets of violence and population (like the study you cited). I suspect there would be techniques for prevention and defense that would cross several (all?) subsets, and many that would not.

 

[Gerry Seymour]

yes it a good example of a study with a VERY limited scope, that therefore can't be just applied to areas out side that scope.
as you seem to indicate that you dont know of any studies with a larger scope, I'm wondering what empiric value that study has to anyone who isn't female, a first year student and lives in Canada, perhaps you could explain?

Those results are likely to apply in similar populations. So, in many (all?) areas of North America, perhaps parts of Europe, where culture isn't drastically different (especially as pertains to women). Age may be a factor, and there are specific risks faced by college students, so some subset of the effective techniques probably don't have the same efficacy everywhere outside that specific population - though they probably do apply to similar populations. I'm not sure we can draw a lot of wide-ranging conclusions regarding men from that study. I'm not sure the drivers of sexual assault with male victims are even the same (not a topic I've ever studied).

 

[Gerry Seymour]

but the extremely limited scope is a weakness, please explain how this can be viewed as a strength

The limited scope is what eliminates confounding variables. Most scientific studies have very limited scope of population/demographics for just that reason.

 

[Buka]

I think opinion is based on personal experience with the subject matter, information, whether the information is sound, and how someone interprets and utilizes the information. But that's just my opinion.

I believe there are other factors involved as well. I was taught not to lead with a rear hand uppercut if you're not close. Too easily countered. Yet, I've seen people get away with using a lead uppercut, and I've done so myself a few times. Had I not been taught that, I would have probably figured it out for myself against really good strikers as I'm sure I would have gotten cracked a lot. But then there's this - what if I had never fought nor trained with really good strikers? Maybe I'd be getting away with far-away lead uppercuts all the time. And maybe, in my little world, that would have been peachy swell.

Now I'm confused. Both anecdotally and empirically.

 

[drop bear]

If the empirical evidence is sound, it should carry more weight than anecdotal. If the empirical evidence is flawed or insufficient, the anecdotal evidence (if there's a significant amount of it) may be of similar weight.

At the very least, the anecdotal evidence should be used to challenge conclusions from empirical evidence, to make sure we aren't either shopping for statistics, or ignoring less-likely-but-not-improbable outcomes.

I have seen this play out though. And anecdotal only counts when it supports your position.

 

[drop bear]

I think opinion is based on personal experience with the subject matter, information, whether the information is sound, and how someone interprets and utilizes the information. But that's just my opinion.

I believe there are other factors involved as well. I was taught not to lead with a rear hand uppercut if you're not close. Too easily countered. Yet, I've seen people get away with using a lead uppercut, and I've done so myself a few times. Had I not been taught that, I would have probably figured it out for myself against really good strikers as I'm sure I would have gotten cracked a lot. But then there's this - what if I had never fought nor trained with really good strikers? Maybe I'd be getting away with far-away lead uppercuts all the time. And maybe, in my little world, that would have been peachy swell.

Now I'm confused. Both anecdotally and empirically.

Or that single hand guard pass that was all the rage for a while.

 

[Steve]

your just taking in riddles trying to wevel yourself out of the corner your in.

you posted that to prove the importance of empirical evidence, but it doesn't seem to so that at,all, so I'm wondering what your point is

Solve the riddle, then. You're a clever guy. I'm sure you can do it.

 

[Steve]

As with any conversation, we have to choose something. We can't discuss "engineering" without agreeing what falls within that area (do industrial engineering, software engineering, etc.?).

Generally, one of two approaches is pretty successful, unless the definition of the term is the actual point of discussion. If you posted first, I usually try to work with your definition, noting that mine is different (to avoid confusion in other threads/discussions). Or, you and I take hold of whichever best facilitates the discussion (most expansive or most restrictive). That requires some cooperation, though. Some folks will simply not cooperate. There's plenty @Steve and I disagree on, but we usually manage to agree on the usage of terms in any given discussion (again, unless the usage of terms is what's under debate).

Of course, if I want to be difficult, I can simply keep using my definition to show flaws in your statements (which there will be, since your statements were made with a different definition). So, if you are talking about Jeeps (uppercase, the brand) and their properties, I could keep insinuating statements about other vehicles commonly included in the "jeep" (lowercase, casual usage, not the brand) category. Every error I point out would be factually correct, but inappropriate to your actual statements. This is one of the most infuriating types of strawman arguments (and something I see on MT from time to time).

For the record, I don't believe @gpseymour disagree on many things at all. A few things, perhaps, but nothing major. And I appreciate that you almost always come around in the end.

 

[jobo]

Those results are likely to apply in similar populations. So, in many (all?) areas of North America, perhaps parts of Europe, where culture isn't drastically different (especially as pertains to women). Age may be a factor, and there are specific risks faced by college students, so some subset of the effective techniques probably don't have the same efficacy everywhere outside that specific population - though they probably do apply to similar populations. I'm not sure we can draw a lot of wide-ranging conclusions regarding men from that study. I'm not sure the drivers of sexual assault with male victims are even the same (not a topic I've ever studied).

the biggest problem with that study, is that they have divieded the population in half, trained one half and not the other, then compared the rapes/ assialts that happened, what not clear, is if training one half made. The other half more vulrable, ie the name number of rapes occurred, but were more in one half of the population, making the training seem more valuable, if the whole group was trained, then it may be that the number of rapes would still be at the same level.as all would be the same level of vulrability . You would also have to look at the selection for each group, as there may be bias there

you need to look at the stats for previous years. And then the,stats for a following year where all have been trained to reach any meaning full conclusion

 

[jobo]

It isn't, by my definition, but it certainly fits within the same category as having your keys in your hand when you approach your car, which some would consider part of the self-defense arena.

I do that as I can't open the,door with out them, if I wanted a weapon is go for something a bit better than key

 

[Steve]

I'd put even a finer point on that, Steve. It's like saying chemotherapy works for lymphoma, and concluding that everyone who has cancer should be treated with chemotherapy.

We'd need to study subsets of violence and population (like the study you cited). I suspect there would be techniques for prevention and defense that would cross several (all?) subsets, and many that would not.

I don't think it's even that specific, and by putting a finer point on it, you're being overly generous to self defense training on the whole.

And so, one would be reasonably skeptical of a doctor who insisted on treating every patient, regardless of diagnosis, the same way. I would approach these self defense guys with the same level of skepticism. "You have cancer. Bloodletting is just the thing." "Oh, you have a broken leg. Let's set that with a stick, and then I'll let some of your blood to allow any unhealthy humors dissipate from your body." "Oh, you have chronic headaches? You guessed it."

Being a little less flippant, try this variation of the analogy. If I go to a doctor with a sore back, he will probably give me a cortisone shot, or a prescription for muscle relaxants. If I go to a chiropractor, he will probably crack my back and maybe send me to a massage. If I go to an acupuncturist, I will probably get some needles in my eyelids. In the same way, if I go to a martial arts dojo, the best form of self defense will happen to be what that person teaches.

When in reality, the best form of self defense for that person might not have anything to do with martial arts.

Interestingly, there was a lot of anecdotal support for those guys (I mean the bloodletters), as well. I mean, not everyone who went to the doctor to lose a bunch of their blood ended up dead. Some even recovered fully.

To your point about things that might apply to a broad cross section, I would agree to an extent. I shared my opinions in the "Pillars of Self Defense" thread.

 

[Steve]

the biggest problem with that study, is that they have divieded the population in half, trained one half and not the other, then compared the rapes/ assialts that happened, what not clear, is if training one half made. The other half more vulrable, ie the name number of rapes occurred, but were more in one half of the population, making the training seem more valuable, if the whole group was trained, then it may be that the number of rapes would still be at the same level.as all would be the same level of vulrability . You would also have to look at the selection for each group, as there may be bias there

you need to look at the stats for previous years. And then the,stats for a following year where all have been trained to reach any meaning full conclusion

That bolded part, which you say is a "problem" is exactly the part that makes the results "empirical." Come on, jobo.

And they did look at stats for previous years, and following years, and also had a control group. You skimmed the summary and didn't actually read the full report.

 

[JR 137]

the biggest problem with that study, is that they have divieded the population in half, trained one half and not the other, then compared the rapes/ assialts that happened, what not clear, is if training one half made. The other half more vulrable, ie the name number of rapes occurred, but were more in one half of the population, making the training seem more valuable, if the whole group was trained, then it may be that the number of rapes would still be at the same level.as all would be the same level of vulrability . You would also have to look at the selection for each group, as there may be bias there

you need to look at the stats for previous years. And then the,stats for a following year where all have been trained to reach any meaning full conclusion

The trained half is the experimental group, and the untrained half is the control group. If there's no control group, there's no valid study.

It's like Motrin relieves headache symptoms in 45 minutes. If a group is given a placebo and their headaches go away just as fast, if not sooner, it's safe to say Motrin had no effect. So long as there's sufficient sample sizes of both groups.

 

[jobo]

That bolded part, which you say is a "problem" is exactly the part that makes the results "empirical." Come on, jobo.

And they did look at stats for previous years, and following years, and also had a control group. You skimmed the summary and didn't actually read the full report.

the study group and the control group are interlinked, if the training works then the number of sexual assualys go up in the control group, you can't really have a study where that happens, that doesn't happen with headaches

 

[Gerry Seymour]

I have seen this play out though. And anecdotal only counts when it supports your position.

The nature of anecdotal evidence is that it's never comprehensive, and can only be used as evidence in support of or contrary to a supposition or claim. It never proves, except in disproving absolutes.

 

[Gerry Seymour]

the biggest problem with that study, is that they have divieded the population in half, trained one half and not the other, then compared the rapes/ assialts that happened, what not clear, is if training one half made. The other half more vulrable, ie the name number of rapes occurred, but were more in one half of the population, making the training seem more valuable, if the whole group was trained, then it may be that the number of rapes would still be at the same level.as all would be the same level of vulrability . You would also have to look at the selection for each group, as there may be bias there

you need to look at the stats for previous years. And then the,stats for a following year where all have been trained to reach any meaning full conclusion

And even that won't be conclusive. That's the nature of studies.

 

[Steve]

the study group and the control group are interlinked, if the training works then the number of sexual assualys go up in the control group, you can't really have a study where that happens, that doesn't happen with headaches

I have a headache. I think you're messing with me, jobo.

 

[jobo]

And even that won't be conclusive. That's the nature of studies.

yes your right and this one is particularly dodgy, that's why I'm still wondering what Steve was trying to show with it