I have a continuing interest in patients understanding the statistics that are used to recommend treatments to doctors - and thus to us. So I was intrigued by this post Wednesday at Science Blog:
Clinical trials: Unfavorable results often go unpublishedAs often happens, there's more to learn from the comments than in the original post:
Trials showing a positive treatment effect, or those with important or striking findings, were much more likely to be published in scientific journals than those with negative findings, a new review from The Cochrane Library has found.
"This publication bias has important implications for healthcare. Unless both positive and negative findings from clinical trials are made available, it is impossible to make a fair assessment of a drug's safety and efficacy," says lead researcher, Sally Hopewell of the UK Cochrane Centre in Oxford, UK.
The international team of researchers carried out a systematic review of all the existing research in this area. In addition to showing that negative results were published less often, they found that if these results were eventually published, they would take between one and four more years to appear in journals than studies showing positive results. ...
"No one will publish a paper about an experiment that gave negative results. The problem is that negative results could as important as positive ones (so maybe other researcher won't try the same thing again, for example). I hope the publication industry will soon disappear, and that the strengths and paradigm of the Internet will finally be used also for scientific articles."My takeaway: it's even sketchier than I thought to presume "If it was important information, we'd have read about it." The process of preparing and publishing articles is fraught with potholes and pitfalls.
"The worst part of this very understandable human trait to publish only successes is this: How can we learn from failures if we never hear about them?"
"I think you could argue that publication bias negatively affects the entirety of scientific research, not just clinical trials. As someone who works in the business, I wanted to mention the PHARMA Code, the code of ethics for the industry. Regarding publication it's pretty clear: you must ATTEMPT to publish findings, significant or not. Now, whether a journal editor wants to publish non-significant results is another story entirely."
"I think it is unfair to blame the journal editors. ... The researchers (in any grant-dependent field) may be required by a grantor or a code of ethics to attempt publication, but my guess is that they don't work very hard to produce a high quality manuscript when all they have to discuss are unsuccessful trials or nul results."
I'm not saying we should ditch journals. We should, though, be conscious of what they are and aren't. Certainly not an inherently authoritative source of information – despite the best efforts of their editors.
p.s. Where did I learn of that post? In an online patient-to-patient community – patients empowering and informing each other. Gotta love the Internet and e-patients!