Between The Lines: Finding the Truth in Medical Literature (Review)

“BTL provides the best critique and comparison of observational vs. interventional (e.g., randomized clinical trials) research studies that I’ve ever read. Even evidence-based medicine experts will find something eye-opening in this book.”

Posting a book review on this blog is a first for me. I am making an exception because this compact volume, Between the Lines: Finding the Truth in Medical Literature, by Marya Zilberberg, MD, MPH, provides an expert’s explanation of many critical issues related to health literacy, evidence-based medicine, and changing models of medical research—all issues that are covered in this blog.

At the highest level, Between the Lines tackles the complex issue of uncertainty in medicine. Dr. Zilberberg presents a framework for assessing the strength of medical evidence in a way that anyone with some basic knowledge of statistics can follow. She uses clear examples that explain, for instance, why a medical test with a 5% rate of false positives could yield a 98% chance of a false positive if the known prevalence of the disease is very low. If these numbers sound irrational, then it’s time you either study Bayesian statistics or read Between the Lines.

In fact, Bayesian statistics are what united Dr. Zilberberg and me. We met via Twitter and our first engaged conversation occurred when she commented on David H. Freedman’s article in The Atlantic: Lies, Damned Lies, and Medical Science. David’s article provoked quite a lot of discussion about the state of evidence-based medicine (EBM), at least based on the type of research we currently consider our ‘gold standard’. His article profiled Dr. John Iaonnidis, who is now chief of the Stanford Prevention Research Center at Stanford Medical School. [1]

When Dr. Zilberberg started explaining the effects of heterogeneity in her blog, I knew I had found someone who had the ability to address important statistical topics in a way that could be understood by a broad universe of readers.

In addition, the book is an excellent resource for non-medical professionals who do have some training in statistics. For me—someone who has experience in econometric modeling and has long been an advocate of Bayesian statistics— but has no formal training in epidemiology, I found the book to be a terrific resource for translating mathematical statistics terminology into medical statistics terminology. All I need now is a self-study guide and comprehension test and I think I’ll feel confident in my understanding of concepts in epidemiology. This shouldn’t be a surprise given that Dr. Zilberberg teaches epidemiology.

I highly recommend this concise volume to anyone involved in peer-review or any aspect of medical communications. I’d even go as far as to say it should be required reading for these groups. And for clinicians and those who determine evidence-based guidelines? Well, I know I’d feel a lot more confident in our healthcare system if I thought that most clinicians could answer the 12 questions that Dr. Zilberberg recommends patients ask before accepting to undergo a medical test or procedure (see Chapter 12).

Finally, I’m confident that Between the Lines will be an important addition to core readings for two groups I highly admire: 1) medical librarians and 2) the Society for Participatory Medicine (http://participatorymedicine.org/).

To obtain a copy of the book, which was published May, 2012, visit the Between the Lines website.

---

[1] See: http://alumni.stanford.edu/get/page/magazine/article/?article_id=53345 for a recent article about Dr. Iaonnidis’s work.

Needed: New Collaborative Models in Medical Research

In a talk I gave in 2009[1], I addressed the challenge of turning the enormous quantities of digital data that will be produced by EHR systems into “reliable, usable” information and emphasized the importance of creating new models of medical research and statistical techniques for dealing with these large universes of new outcomes data.   I went out on a limb at the time –since my training in statistics was in business school, not medical school—but have since found many who support my position among the best and brightest in the medical community. 

Weaknesses of current research methods

Currently we use the term “translational medicine” for the process of taking the results from research studies, typically Randomized Control Trials (RCTs), and transforming them into data that can be used in a clinical setting.  The process includes writing articles for publication to describe the research findings in medical journals (after peer review), which then get further interpreted by clinical teams at medical societies/associations/public health authorities to produce guidelines or other clinical applications.  At the same time, published research results get disseminated in the mass media by healthcare journalists.    Many of the difficulties and inadequacies of this approach have come to light as the health IT infrastructure in the US struggles with integrating clinical evidence into electronic health records (EHRs).   The two toughest issues are:  1) the difficulty of extracting data from journals (print or electronic format) because the journal format wasn’t designed for data extraction, integration or machine analysis and 2) the lack of collaboration between research institutions that leads to an enormous number of research results that don’t build on previous studies and are sometimes contradictory.   As a result, even elaborate data mining engines—or a redesign of journal formats—won’t by themselves solve core problems with our current research system. 

New Data, New Models

Beyond lack of collaboration, there is a more fundamental issue about the appropriateness of RCTs as a source of clinical guidance.   Marya Zilberberg, MD, MPH writes in JAMA that we have an “avalanche of published research” yet the “data generated in an RCT are frequently irrelevant because of their limited generalizability” [2].  The cause of the limited generalizability is the “hetereogeneous treatment effect” and I won’t go into details of what that means here; for that, read Dr. Zilberberg’s blog series on Reviewing Medical Literature, which is now at part 5 and is highly recommended reading.   But, as you can probably guess from the word “heterogeneous”, the problem relates to the fact that we don’t all respond to a specific treatment in the same way.

Another recent article published in New England Journal of Medicine (NEJM)[3] buttresses my position that we need new research models and new incentives for collaborating.  This article presents some stark data on how little emphasis – and financial support – has been placed on outcomes research, best practices, new care models, quality, comparative effectiveness or service innovations relative to total biomedical research (0.1% v. 4.5% of total health expenditures).   The 0.1% will increase to 0.3% in 2010 according to the authors, but that’s still a small percentage considering that we are facing a huge pool of new outcomes data that will be generated by EHRs and haven’t yet agreed upon the best statistical methods for organizing and analyzing these collections of data that hold so much promise for enhancing the clinical value of medical research. 

There are models that have been developed for analyzing outcomes data, especially within the payer segment.  Registries of patient data have existed for some time, but have been limited in scope or the  reliability of the data sources (e.g., claims data).   But, in order to achieve a “higher standard” of clinical value, more collaboration and “development of more robust analytical techniques for ascertaining clinical value”[4] are needed.  

This emerging field offers opportunities for data publishers that understand how to apply master data management (MDM) principles and data analytics/predictive analytics companies in biomedical research that can adapt their models for clinical applications. Perhaps the biggest opportunity is for standards organizations and data publishers to build a collaborative infrastructure for better aligning biomedical research and clinical decision support systems. 

---

[1] http://www.slideshare.net/janicemc/epatientconnections2009-health-content-for-epatients

[2] The Clinical Research Enterprise; Time to Change Course?, JAMA, February 9, 2001—Vol 305, No. 6, Marya D. Zilberberg, MD, MPH,: http://jama.ama-assn.org/content/305/6/604.short.

[3] Biomedical Research and Health Advances, NEJM, February 9, 2011, Hamilton Moses, III, MD, and Joseph B. Martin, MD, PhD: http://healthpolicyandreform.nejm.org/?p=13733.

[4] Ibid.