6 Responses

1. e-Patient Dave

   August 3rd, 2008 at 10:03 am

   1

   Great, provocative post, Ted. I’m circulating it in the back room in the e-Patient Scholars Working Group.

   As we shift more and more to patient-centered medicine, and patients in need grab the most useful information they can find, the resulting “marketplace” will show something about what works.

   Here’s something to ponder. As I understand it, the purpose of peer review is to avoid publication of specious or sloppy work, which would have led to incorrect or imprudent diagnosis or treatment. (Please correct me if I’m wrong about that.) Well, there was that landmark JAMA paper in 2000 saying that medical error was the #3 cause of death. Similarly, one could start a tally of “medical errors” caused by failure to use information newer than the latest peer reviewed material.

   As you know, in my case that was no small issue. The kidney cancer patients on ACOR.org often cite encounters with doctors whose information is seriously out of date, compared to the not-yet-published results of clinical trials; and the patient community at PeoplesPharmacy.com gathered data (not found in the peer review process) that finally led the FDA to agree there was a problem with the generic of Wellbutrin.

   As the rate of creation and dissemination of important new information accelerates, will the long slow peer review process become increasingly irrelevant and out-of-date as a gauge of reliability?

2. Ted Eytan

   August 3rd, 2008 at 10:53 pm

   2

   Dave,

   I think this is a concern that people have, and as I talk to patients myself, now realize that within the big chunks of evidence supporting this or that approach, there are hundreds of nuances that are not well studied rigorously, yet maybe 10-20 patients might be able to inform about, based on their experience. If a decision to begin taking a potentially powerful biological substance is at stake, this extra bit of information might be very useful.

   Another concern is that those doing the reviews may have conflicted relationships and/or unclear understanding of who their “customer” is, especially if the work they are reviewing comes from a more senior researcher or a researcher who may have power over them in different contexts. It’s a real issue for which medical journals are doing their best to mitigate,

   Ted

3. e-Patient Dave

   August 4th, 2008 at 2:17 am

   3

   It’s hard to believe it was only 18 weeks ago that the thread you cite happened. Boy have I been exposed to a lot since then. But that was a great thread, well worth re-reading.

4. Ted Eytan

   August 4th, 2008 at 8:31 am

   4

   So we’re still a little ahead of our time? It’s all about the journey, in the company of those we serve,

   Ted

5. Dan Hoch

   August 7th, 2008 at 9:28 am

   5

   Ted and Dave,
   I am especially interested in this topic, since like Wachter, am trying to survive in academia. I think you’ve hit two issues on the nose, but only touched on a third. First, you’ve alluded to the incentives for those trying to publish. We want as many people to read our ideas as possible. Thus, hurdles, like peer review and the cost of publication (and subscriptions that inhibit many people from reading our work in mainstream journals) are problematic. Secondly, you touched on the the issues of fraud, conflict of interest and related problems that peer review and journal policies try to contain. Can internet-style peer review serve a similar function? Probably. But, I think there is a potential problem in a different kind of manipulation of our knowledge related to the problem of conflict of interest, but slightly different, that is the problem of the “free market of ideas” and “marketing”.

   As we’ve see in our political and economic realms over the last 20 years, the invisible hand of the “free and unregulated market” is not so perfect after all, whether we’re talking about the market for corn, or for ideas. First of all I’m of the belief that there is no such thing as a truly free market. There are always influences other than supply and demand that set price and impact quality. We live in a society of laws, and they have consequences. Secondly, the tendency for markets to centralize and and concentrate power and money is pretty clear now too. Thus, if my agenda is to “market” my ideas, and I have enough skill and knowledge about how the internet works, and perhaps a little extra money, I can make my ideas pretty prevalent no mater how far out of the mainstream and how “wrong” they may be.

   I don’t see peer review at journals as the only check on this problem, but it helps. I don’t know the best way that this can be controlled on the wild frontier of internet knowledge sharing, so will quit writing and see if anyone else has an idea

6. Ted Eytan

   August 7th, 2008 at 9:47 am

   6

   Dan,

   Awesome comment and thanks for both (1) adding balance to the discussion and for (2) being part of the solution since you are beholden to this system and pushing the edges from within.

   Perhaps there might be a side-by-side marketplace of credibility/authority that go along with a new model, measured by a combo of real-time ratings by both physicians AND patients who did/did not benefit from the knowledge (let’s make this democratic), and verification of conflict of interest (or lack thereof). I can’t help this would be better than what we have today. It worked for eBay.

   For my part, I have just added a statement to my About page about my not having any ties to the pharmaceutical or device industries, and I would gladly submit (even for a sum of money) to a third party evaluation of such (is there a place where I can do this?),

   Ted