Critical Reading of the Medical Literature

When articles are presented at Journal Club, when you read articles in journals, or when articles are reviewed in the lay press, questions should arise in your mind about whether the study is appropriate, the data is real, and the analysis is appropriate. Every piece of medical literature is not law simply because it is published. Editors do the best job that they can but, in reality, the true value of individual articles in the medical literature can only be surmised after post publication analysis and time.
It takes many years to develop the knowledge and skill to analyze journal articles. I hope that, with our departmental journal clubs, we can move this process along so that all can be appropriately critical of the information that we are presented with.
These are some of the questions that come to mind as I peruse journals every month:
  1. Are the authors asking a question that can reasonably be answered? – Basic scientists ask this question every day. There are some things that we would like to know that, at the present time, do not lend themselves to randomized control trials. This relates to the need for total disclosure of risks and benefits to patients, to the physiologic differences between physiologic models in animals and humans, and to the lack of a profit motive to drive the asking of specific questions. Because of these limitations, there are some important questions that that will never be answered through analysis of multiple randomized controlled trials.
  2. Is the study that is designed to answer the question methodologically appropriate? –This is a broad and difficult topic but relates largely to determining whether there is bias underlying the development of the trials. This may be difficult to determine because even with the requirements for documentation surrounding the preparation and implementation of RCTs there is much that is unknown. That said, industry funding of RCTs often produces blatant or unwitting bias, both in the production of trials and in the non - publication of negative trials.
    Blinding of those carrying out trials is critical to accurate determination of an effect in a RCT. The sophistication that the investigators have in blinding those that are carrying out the trials is an important determining factor that should be considered as you read the article.

  3. Is the statistical analysis of the data derived from the trial appropriate for the question that is being considered? Neither an unpaired T test nor Chi square analyses will answer all questions. Lucky for us, most of the major journals employ PhDs in Statistics to assist. 
  4. Is the information that is being presented new information, old information, or does it contradict information that we think we know? – Surprisingly, all of these are important. Knowing that a positive result can be replicated is important because it adds weight to our ability to incorporate information into our practice. It helps to answer the question “Is this information generalizable?” If an article contradicts accepted information it implies that further study of the old information and the new information is required. Did the old trials have inherent bias that was originally missed?
  5. Does (or should) this study have an impact on my practice? – Will I do anything differently based on the results of this study? Does it validate what I am currently doing? In my opinion, changing practice based on a single study should be a rare event during a professional career.

As time passes, you will undoubtedly add to these questions, and many other interrogatives can be substituted. However you read the medical literature for the rest of your career it is important that this reading be systematic and critical.

Rae Brown, M.D.

Comments

Popular Posts