John Grohol, Psy.D., is founder and publisher of PsychCentral, a pioneering community of e-patients. After he read my post the other day about evidence-based medicine, he sent me a paper worth reading: Helping Doctors and Patients Make Sense of Health Statistics.
This is relevant to the e-patient movement because as you and I become more responsible for our own healthcare, we need to be clearer about what we're reading. Plus, it appears we could be more vigilant about what our own professional policymakers are thinking.
The paper is 44 pages, but even the first few will open your eyes to how statistically illiterate most of us are - and that includes MDs.
Consider this question, which was given to 160 gynecologists:
Assume the following information about the women in a region:21% of them got the right answer (#3, 1 chance in 10). 60% guessed way too high, the other 19% guessed #4. (That's 10 times too low).
A woman tests positive. She wants to know whether that means that she has breast cancer for sure, or what the chances are. What is the best answer?
- The probability that a woman has breast cancer is 1%
- If a woman has breast cancer, the probability that she tests positive is 90%
- If a woman does not have breast cancer, the probability that she nevertheless tests positive is 9% (false-positive rate)
- The probability that she has breast cancer is about 81%.
- Out of 10 women with a positive mammogram, about 9 have breast cancer.
- Out of 10 women with a positive mammogram, about 1 has breast cancer.
- The probability that she has breast cancer is about 1%.
The paper presents numerous other examples of statistical illiteracy (an example of "innumeracy"), misunderstandings of data that lead to serious unintended policy consequences. My personal favorite is the opening item about Rudy Giuliani's assertion that he's lucky to have gotten prostate cancer here instead of under the UK's "socialized" medical system. It's not because I don't like Giuliani - it's that his own misunderstanding of the data he was quoting led him to advocate something that had nothing to do with his actual odds. He himself would have been harmed if he'd been guided by his own best advice. And he's not alone in that.
The paper proposes uncomplicated ways to improve our comprehension. First among them is to stop talking in percentages and talk instead in raw numbers. Phrased that way, the same three facts that were given to the gynecologists is much clearer:
- Ten out of every 1,000 women have breast cancer
- Of these ten women with breast cancer, 9 test positive
- Of the 990 without breast cancer, 89 nevertheless test positive.
Another example echoed what The End of Medicine said about Lipitor. (Without Lipitor, 1.5% of the control group had a coronary event; with Lipitor, about 1% still had one.) A 1995 alert in the UK warned that certain oral contraceptives doubled the risk of blood clots in the lung or leg. Understandably, many women stopped taking the pill; within three years, 13,000 more abortions were performed, reversing five years of decline, and there was a matching increase in live births.
What was the risk that led to this? In raw numbers, one woman in 7,000 has such a blood clot anyway; with this pill, one more blood clot happened.
The irony in this case is that both abortion and childbirth carry more risk of clots than the pill itself. In other words, one benefit of the pill is that it avoids the risk of clots associated with the end of any pregnancy.
So although the number presented ("double the risk") was absolutely accurate, the real clinical impact wasn't nearly as absolute.
This is a taste of what's in the first few pages. It gets dry in places but even the first few pages are compelling and informative - and at no point does it require that you be a mathematician. The explanation of Giuliani's error is particularly good.
Thanks to the good Doctor John for the link.
Continued in part 2