I am always delighted to meet other physician bloggers, and such was the case with Bob Wachter, MD, who’s a physician that blogs, and from within academia. That’s rare, and welcomed, by me for sure.
He recently wrote this piece, Will Knols and Blogs Upend the Cozy World of Medical Publishing?, which echoes several ideas I have been having since I started blogging, especially around the idea of, as he calls it, “the democratization of peer review.” (I’m focusing on the comments about medical publishing – the world of Knols appears to be experiencing a rocky ride)
As I occasionally get requests to write for peer reviewed journals or books or I consider writing for them, I have been pausing to ask, “why?”
This is especially when the difference is between instant access and feedback to the people I serve, versus a smaller group of individuals with (potentially) limited experience in the ideas I’m writing about, and the medium I use to write them in. I say this without any predjudice to the publishing community – I am just not sure where physicians in my generation will fit in moving forward, unless the model is changed.
Bob refers to the difference between “Having an article peer reviewed by 3 experts is different than having 17 Joe Six-packs;” however, I’m not sure what the difference is, depending on the issue, between those two constituencies – what’s an expert in if she/he isn’t a person “just like me?” Also, what’s the value of a single (relatively speaking), private, review, that will be locked in time and space, forever? Robert Scoble speaks well to this in the post “Scoble Defends Blogging (Again), and He’s Right (Again).”
I do not work in academia, where people are incentivized/rewarded for the number of peer-reviewed publications with their names on them. I think a deeper question that should be asked, is, “What’s the best way, in this millennium, to produce portable knowledge that can be used by others?” I have talked with innovators in academia who have not shared their knowledge because of the effort required to publish to medical journals. That’s unfortunate.
How could the reward/incentive system in academia be reconfigured to respect the many different ways people can share knowledge, and put them to use to help people? I think it could be, and in turn a lot of great ideas could be unleashed.
Bob mentions in his post that he submitted his piece to two medical journals, who rejected it. However, we still get to read it thanks to Web 2.0.
I’m not even going to try with this one. And I sort of don’t have to.
7 Comments
Great, provocative post, Ted. I'm circulating it in the back room in the e-Patient Scholars Working Group.
As we shift more and more to patient-centered medicine, and patients in need grab the most useful information they can find, the resulting "marketplace" will show something about what works.
Here's something to ponder. As I understand it, the purpose of peer review is to avoid publication of specious or sloppy work, which would have led to incorrect or imprudent diagnosis or treatment. (Please correct me if I'm wrong about that.) Well, there was that landmark JAMA paper in 2000 saying that medical error was the #3 cause of death. Similarly, one could start a tally of "medical errors" caused by failure to use information newer than the latest peer reviewed material.
As you know, in my case that was no small issue. The kidney cancer patients on ACOR.org often cite encounters with doctors whose information is seriously out of date, compared to the not-yet-published results of clinical trials; and the patient community at PeoplesPharmacy.com gathered data (not found in the peer review process) that finally led the FDA to agree there was a problem with the generic of Wellbutrin.
As the rate of creation and dissemination of important new information accelerates, will the long slow peer review process become increasingly irrelevant and out-of-date as a gauge of reliability?
Dave,
I think this is a concern that people have, and as I talk to patients myself, now realize that within the big chunks of evidence supporting this or that approach, there are hundreds of nuances that are not well studied rigorously, yet maybe 10-20 patients might be able to inform about, based on their experience. If a decision to begin taking a potentially powerful biological substance is at stake, this extra bit of information might be very useful.
Another concern is that those doing the reviews may have conflicted relationships and/or unclear understanding of who their "customer" is, especially if the work they are reviewing comes from a more senior researcher or a researcher who may have power over them in different contexts. It's a real issue for which medical journals are doing their best to mitigate,
Ted
It's hard to believe it was only 18 weeks ago that the thread you cite happened. Boy have I been exposed to a lot since then. But that was a great thread, well worth re-reading.
So we're still a little ahead of our time? It's all about the journey, in the company of those we serve,
Ted
Ted and Dave,
I am especially interested in this topic, since like Wachter, am trying to survive in academia. I think you've hit two issues on the nose, but only touched on a third. First, you've alluded to the incentives for those trying to publish. We want as many people to read our ideas as possible. Thus, hurdles, like peer review and the cost of publication (and subscriptions that inhibit many people from reading our work in mainstream journals) are problematic. Secondly, you touched on the the issues of fraud, conflict of interest and related problems that peer review and journal policies try to contain. Can internet-style peer review serve a similar function? Probably. But, I think there is a potential problem in a different kind of manipulation of our knowledge related to the problem of conflict of interest, but slightly different, that is the problem of the "free market of ideas" and "marketing".
As we've see in our political and economic realms over the last 20 years, the invisible hand of the "free and unregulated market" is not so perfect after all, whether we're talking about the market for corn, or for ideas. First of all I'm of the belief that there is no such thing as a truly free market. There are always influences other than supply and demand that set price and impact quality. We live in a society of laws, and they have consequences. Secondly, the tendency for markets to centralize and and concentrate power and money is pretty clear now too. Thus, if my agenda is to "market" my ideas, and I have enough skill and knowledge about how the internet works, and perhaps a little extra money, I can make my ideas pretty prevalent no mater how far out of the mainstream and how "wrong" they may be.
I don't see peer review at journals as the only check on this problem, but it helps. I don't know the best way that this can be controlled on the wild frontier of internet knowledge sharing, so will quit writing and see if anyone else has an idea
Dan,
Awesome comment and thanks for both (1) adding balance to the discussion and for (2) being part of the solution since you are beholden to this system and pushing the edges from within.
Perhaps there might be a side-by-side marketplace of credibility/authority that go along with a new model, measured by a combo of real-time ratings by both physicians AND patients who did/did not benefit from the knowledge (let's make this democratic), and verification of conflict of interest (or lack thereof). I can't help this would be better than what we have today. It worked for eBay.
For my part, I have just added a statement to my About page about my not having any ties to the pharmaceutical or device industries, and I would gladly submit (even for a sum of money) to a third party evaluation of such (is there a place where I can do this?),
Ted
When worlds collide!!
I produce a podcast for The Hospitalist, and am very familiar w/Bob Wachter’s outstanding work in building the Society of Hospital Medicine and the hospitalist specialty. One of my main concerns as a health writer/blogger is to make sure I’m not carrying anyone else’s water – i.e. I’m often too willing to bite a hand held toward me with food (in this metaphor, that would be $$) if I suspect that I’ll be expected to flack some version of 21st century snake oil.
Thought-leadership became a dirty phrase thanks to pharma’s relentless quest for MDs to pimp their wares. Not that what pharma offers isn’t often life-saving stuff, but then there’s restless-leg syndrome and the huge cohort of un-replicable “scientific” research that gets published annually. @IvanOransky and his RetractionWatch.com site go a long way toward clearing the fog on the life-sciences battlefield …