More on statistics: deadly omissions, deadly conflicts

It appears that my inquiry into statistics, which I started here and cross-posted onto the e-patients blog, has started something of a storm. I am amused and pleased - after all, the only thing I ever really wanted to do was turn the world upside down, and this is as good a start as any.

(Okay, any of you who think "That doesn't sound very modest, deB," give it up. This is MY blog, and I'm happy to have started something that I think is productive. :))

Here's how the blogosphere works.

First, I wrote about evidence-based medicine, just because I'd been mulling the subject after reading Anticancer. Then John Grohol, one of the e-patients.net authors, sent me a paper, which I wrote about here.

I cross-posted it on the e-patient blog. The timing was great, because that morning the NY Times piece came out, misreporting the new evidence on statins. (Note: I'm not saying they made false statements - I'm saying they didn't give us the info we need.)

The comments on that post led Dr. Ted Eytan to point out a great web site that evaluates the quality of health news reporting, HealthNewsReview.org. Here is their comment on the reporting (in the LA Times) on the statins story.

Gilles Frydman, founder of ACOR, dug into it and wrote Lies, Damn Lies And Statistics: Collective Statistical Illiteracy. (He did some digging and found that the first known publication of that expression was in reference to medical research!)

The Psych Central blog picked up Gilles's post.

Gilles continued Thursday, partially inspired by the issues raised by HealthNewsReview, and wrote the slash-and-burn What's wrong with this picture? He pastes in the conflict of interest fine print that appears at the end of the article: 345 words of disclosures about how the article's authors receive funds from the makers the medications described in the article. And the disclosure then ends with the delightful phrase: "No other potential conflict of interest relevant to this article was reported." [Emphasis added.]

And this was no tabloid rag - this is in the New England Journal of Medicine, one of the most respected journals in the world.

What would you think about news reporting by writers who are literally being paid by the people they're reporting on?

I think I'm going to make a point of keeping an eye on HealthNewsReview, and maybe start bitching at the NY Times when I spot them omitting vital information like that.

Anyway: in the comments on that post, Sarah Greene, who recently left the NY Times online health department, speaks from the perspective of someone who's been on the inside of a big modern news organization. In her comment she remarks, "More to the point, patients don’t get to wear that E-badge if they aren’t first Educated to understand Evidence."

She adds, "my shock-disbelief-sadness centers on the media. Can journalists be held accountable at a time when news organizations are pushing for more & faster & shorter, coupled with desperation for digital advertising dollars?"

All in all, I agree with Sarah's remark that it's up to you and me to dig into the numbers when a treatment is recommended. As Making Sense says, if we don't know the actual numbers, the whole concept of "informed consent" doesn't work.

And, sadly, a new contributor, Karen Vaughan, speaks up from the diabetes community and highlights something I hadn't noticed: "About 20% more people got diabetes in the drug group. For every life 'saved' by the drug, someone else got diabetes. Factor that into the 'costs' of the drug." That issue too is reported in Making Sense: every treatment (or decision not to treat) has side effects, and no informed choice is possible unless all that is laid out.

Vaughan answers the balance question, concluding: "If a person took a 30 minute walk per day they would have better results without the side effects or costs."

Wouldn't you think those alternatives should be laid out for our consideration in a journal that purports to inform the medical trade about how best to achieve health? I don't mind reporting on new medications, but simple decent scientific method requires documenting all the known, relevant pro's and cons.