Saturday, November 28, 2009

How to evaluate something like the mammography recommendations

This is a long post because it's not a simple topic. Please don't jump to conclusions.

On the e-patients blog, Gilles Frydman, founder of ACOR, has written one of his potent deep-thinking posts. This time he reflects on how we (humanity) tend to get ourselves in trouble when we frame our thinking in terms of "war on..." It doesn't matter whether it's a skillful manipulation ("war on terror" / "Iraq - 9/11") or unintentional; fear comes from the more primal parts of the brain and can prevent that distinctly human skill, thinking.

This is adapted from a comment I posted.


Before I start, let me say that I have not evaluated or even read the task force's recommendations. The instant I heard the public reaction I knew there was craziness in the air: insanity, reactions to things that weren't being said, hysteria. And I knew what I'd write about wouldn't be a thumbs-up-orodown post, but how to approach such a consideration as an informed, engaged e-patient.

What does this have to do with being an e-patient? Everything. It's increasingly clear that we can't rely on the establishment (political leaders and news media) to interpret science correctly. Gilles lays out the evidence for that well. We must learn to interpret evidence for ourselves.


1: The purpose of using statistics

Gilles, thanks for tying this to our previous posts on understanding statistics. PLEASE, people, read last November's short post on understanding statistics. And consider reading the excellent paper it links to.

Look at its incredible evidence that even physicians misinterpret statistics - the example on the second page is specifically about gynecologists being unable to correctly interpret mammography data, unless it's presented quite clearly. The point isn't that docs are dumb, it's that misinterpreted statistics lead to confused, botched conclusions.

See, the purpose of statistics is to improve our guesses when we don't have certainty. Use the tool correctly and it'll help; use it wrong and it'll drive you right off the road into a ditch.

Another example in that article is Rudy Giuliani's ignorant, erroneous comparison of apples-and-oranges prostate screening data from the US and UK - data collected under different circumstances in different environments at different times in life. The reality is that prostate mortality is the same in the two countries, but Giuliani's bogus logic led him to declare that the US healthcare system works better than the UK's.

Right: same mortality rate, but pick some arbitrary statistic and claim that it shows there's a difference, even when the actual mortality rate is the same. Brilliant civic leadership.


(Not to be outdone, in the UK, Tony Blair declared that his country was doing poorly and must improve 20%.)

STOP BEING IGNORANT. Don't use a tool if you don't know what it's for!

In my annual physical today, Dr. Danny Sands and I again discussed whether I should get a PSA test (prostate cancer test). He first brought it up three years ago, and he (unlike many physicians) made clear that the PSA is notoriously unreliable: many false negatives, and many false positives that lead to unnecessary treatment. That's the kind of advice I like: give me straight-up information about the pros and cons, and let me choose.


2: Every choice in life has risks.
You cannot evaluate doing something without also evaluating not doing it, and comparing.

See, as with the prostate example, there are risks to screening and risks to not screening.

Many articles have discussed that screening tests (a) cost money, (b) can lead to unnecessary treatment because of false positives, which (c) cost more money and (d) can cause harm.

It's exactly the same as the UK birth control issue. The erroneous public reaction was caused by evaluating one arm of the choice and not comparing it with the other. Result: greater risk of clotting: a botched conclusion with medical consequences. You could call it statistical malpractice.

Just SHUT UP if you want to tell me "Don't take away my tests!!" I didn't say that. Don't be insane, hearing things I didn't say. That's hysteria.

I'm not talking about the conclusion, I'm talking about how to make intelligent choices.


3. Who gets hurt? The patient.

In a comment on Gary Schwitzer's excellent blog, I said what really upsets me in the mammography insanity is that (ironically!) it's women who are put at increased risk by these misinterpretations: women who are left with less accurate advice.

In a similar way, the first case in that paper is about a 1995 UK scare caused by news that new birth control pills were causing a 100% increase in blood clots. True – but the reality was that they produce 2 clots per thousand women, vs. 1 for the old pills. Literally a one-in-a-thousand difference. But oh wow, that's 100%! Headlines!

Here's the consequence: Because of ignorant misinterpretation, massive numbers of women went off the pill, and in the following year ...
  • 13,000 more abortions were done in England and Wales
  • 13,000 more births, including 800 under age 16
The special irony is that both abortion and pregnancy have higher rates of clotting than the pill itself. So the whole ignorant diversion was not only ill-considered, it worsened women's health (while increasing abortions and births).

Doesn't it irk you that the righteous protests about protecting women's health, ignorantly considered, lead to bad advice?? These errors in interpreting science harm people.


4. The idiocy of asking celebrities their opinions

Whatever credibility our network news media had with me, they lost it when they trotted out ignorant celebrities to contribute to the misinformation campaign.

A new blog, the oddly named Celebrity Diagnosis, has chosen the mission of talking about health issues through the lens of celebrity discussions. They cite how dumb celebrities have been on this topic, particularly Jaclyn Smith saying "They want to abandon proven therapies. It's wrong."

Excuse me? Since when is a test a therapy? Jaclyn, is checking your pants the same as laundering them? Maybe that's over your head. So shut the hell up on medical advice, willya?

But that's what Gilles is talking about when he cites moral panic, which the Center for Media Literacy defines as "A sudden increase in public perception of the possible threat to societal values and interests because of exposure to media texts."

And what better way to drive people to panic than fear of death?

But hold on; I've been there. Let's think.


5: On the fear of cancer and death

When I learned on 1/22/07 that my median survival time was 24 weeks, it was NOT pleasant for me. "No. I am not done. Not ready to go."

But after the abrupt shock, I found myself able to look it in the eye and think. What are my choices? Where do I start?

Twenty years ago I listened to tapes titled "Conscious Aging" and "Approaching Death" by Ram Dass, a spiritual teacher who was born Richard Alpert and who, in the sixties, wrote Be Here Now. After years in Tibet with gurus he worked with many dying people during the AIDS epidemic. (I've seen him speak, and believe me, when he entered the large hall, some sort of presence filled the room. Never seen anything like it.)

He speaks of death matter-of-factly, and his perspective aided my acceptance of what I was abruptly facing. I wasn't interested in going through that door, but after a while I realized that if that's what was happening, I could face it.

And that freed me to have authentic conversations with people about it, and with myself: I was able to choose my path with a clear mind.

I don't diminish the concerns of people who face cancer and/or death - everyone's trip is different. But throughout my cancer journal on I talked about the power of the words and concepts we use in discussing our world. I opted not to engage in discussion of "OMG!" and "How terrible!" I chose to think about "What are my options? What could be done that would make any difference?"

If we could all face death calmly I bet we'd be able to make much better choices in how we interpret evidence.


6: Think.

That's why hysteria (Wikipedia: "unmanageable fear or emotional excess") is so very, very harmful. When people lose their minds, perhaps because they're told foolish misinterpretations of the evidence, what are the odds of reaching a sane conclusion?

Oh wait: we don't have to ask the odds. This time we know.

Well done, Gilles. Obviously you touched a nerve.


  1. Simply put: Excellent post!! Thank you for bringing a wider lens to the discussion.

  2. Lisa and everyone, keep an eye on the other comments on the original post.

  3. I question the conclusions of groups like USPSTF. There is a whole series of events intervening between screening, biopsy, detection of possible cancer, sometimes early stage, decision making in collaboration with physicians, possible treatment, outcome. Missteps in any one of the stages can lead to an undesirable outcome EVEN IF screening provides a key starting point (which so many of the experienced experts believe --- no matter what the statisticians conclude). So why are the statisticians wrong? Because they ignore the path. For example, a report has just come out , showing that the majority of prostatetectomies in this country are performed by surgeons with extremely limited experience resulting in a substantial reduction in effectiveness and increase in negative side effects. But wait you say, what does that have to do with PSA screening. Simple --- many of the men getting inferior treatment had their initial diagnosis via psa screening. So a study by statisticians which just looked at PSA and at final outcome would "conclude" that the PSA screening didn't work, when what really didn't work was a further step -- the surgery because of the incompetence of the surgeons. USPSTF needs to look at these sorts of factors before coming up with much in the way of useful conclusions in my opinion, or else all they should conclude is that you don't have good followup available don't get mammograms and PSA tests. More importantly --- we need advice on how to get qualified practitioners, not on avoiding obtaining potentially useful knowledge about whether or not we have early stage cancer. From the report:
November 25, 2009 — The majority of surgeons performing radical prostatectomy in the United States have extremely low annual caseloads, which can result in an increased risk for surgical complications and cancer recurrence, according to a new analysis published in the December issue of the Journal of Urology.

    Lee Smith

  4. Dave, I think your point about statistics is extremely well taken. There is grossly inadequate education in statistics in our country, among both laymen and physicians. I do not recall ever being specifically trained in how to evaluate the statistics in a medical article, although those who are in academics and do this regularly are proficient by OJT (on the job training).
    I believe the entire public should receive a full year's training in statistics as part of high school. I unsuccessfully lobbied for this for my own daughter - but was overruled by a husband who thought calculus was more important. When was the last time you used calculus in your daily life?!! (ps she had already had standard calculus anyway, this was an argument over AP Statistics vs. AP calculus)

    nonlocal MD

  5. Nonlocal,

    What a great idea - teaching statistics in school!

    You might like the great new training materials Gary Schwitzer has on his revamped Health News Review site. See the "Our Criteria" list on the right side - most of them have video interviews with me. :)


Your comments will be posted after they are reviewed by the moderator.