How to Critically Read a Research Study

As clinicians, it’s vital that we stay current with the literature. Countless scholarly articles are published every day, and it’s important that we are able to critically analyze and integrate relevant new information into practice. Read on to learn two different approaches to reading a scientific paper.

---

Reading a scholarly article with a critical eye is not an easy task. In this article, I’ll talk about how to approach critically analyzing a research study and what to keep an eye out for.

First, Decide If It’s Worth Reading!

Ever spend half an hour poring through a study only to get to the end and realize it didn’t contain any information that was novel or useful to you? Every day, 11 new systematic reviews and 75 medical trials are published. This number is only increasing. (1) With so many studies out there, we really need to focus on reading the most salient articles.

Luckily, the abstract provides a quick synopsis that includes the purpose of the study, the most significant findings, and the authors’ conclusions. Before you decide to critically review an article, skim through the abstract to see if it will be useful to you. What are the major ideas? Who were the subjects? What variables were assessed? What were the main findings? Get an idea of what’s to come, but don’t be tempted to just read the abstract: it’s always important that you look at the data yourself and read the full text.

Next, note a few other article characteristics. What journal is the study published in? When was it published? Who are the authors, and what are their affiliations? Flip to the end of the paper and look at the funding source and any potential conflicts of interest. Was this funded by the sugar industry? Knowing this information in advance can help you keep an eye out for biases throughout.

Learn how to spot bias and poor design in research studies

Approach #1: Simple Front to Back

The first approach is to simply read the journal article front to back, in the order that the authors have presented it. This approach works well if you have very little background knowledge about the topic and is great for those who have little experience reading scientific papers and are just getting started.

The introduction of the paper will provide any necessary background information and introduce the rationale for the study. You’ll want to ask yourself: what are the gaps in the literature? What did the authors set out to test, and what was their hypothesis?

You can then usually skip the methods and head right to the results, where the authors will walk you through each figure. If the article is well written, the results section should be understandable without having read the detailed methods section. Feel free to refer to the methods section if necessary to clarify what the authors did, but if it’s not your area of expertise, avoid getting bogged down in unnecessary detail. (For example, if you don’t know what polyacrylamide gel electrophoresis is, it’s probably not worth your time to try to determine whether the researchers performed it correctly.)

Finally, the discussion is where the authors will summarize the results, talk about how the study fits in with other related literature, and offer future directions and implications of their findings. Do the authors acknowledge limitations? Do they provide an unbiased assessment of other literature?

This approach will give you a complete idea of what the researchers did, what they found, and why it might matter. However, it is not the most conducive to critical analysis. As you get more accustomed to reading literature, you’ll want to move towards approach #2.

Approach #2: Results First

The second approach works especially well if you are familiar with the topic. It is also the approach you will want to gravitate toward as you get better at reading papers because it allows you to take a critical eye to the data before being influenced by the authors’ interpretation.

In this approach, you start by skimming the last paragraph of the introduction, before heading right for the results section. This last introductory paragraph typically provides the authors’ rationale for the study and the hypothesis they are trying to test.

You can then jump right into the figures and try to make sense of the data without reading through the text of the results. The main idea here is that you want to avoid reading the authors’ interpretation of the data until you have developed your own opinions. You’d be surprised by the number of published articles in which the text of the results skews or even blatantly contradicts what was shown in the figures. So, work your way through the figures, using the figure legends to help you make sense of things, and flip to the methods to clarify things as necessary.

Only when you have a grasp of the figures and methods should you read the text of the results. See if your interpretation of the data matches what the authors interpreted. Lastly, read the discussion to see if you agree with how this study fits in with the literature. Are there any studies they left out? If you are very familiar with the topic, you can also scrutinize the methods to see if you agree with their methodology.

Understand the Study Design

Was the study performed in humans, animals, or cell culture? Is it an observational or experimental study design? Were the researchers blinded? Were the participants blinded? Was it a short-term or long-term study? What variables/endpoints are they testing? All of these questions should be on your radar when reading through a primary literature article. Understanding the design of the study allows you to be on the lookout for various limitations that are characteristic of different designs. It also can help you to determine the strength of the evidence.

Look for limitations and biases:

• Self-reporting bias: very common in nutritional science research
• Unhealthy user bias: discussed in this podcast
• Confirmation bias
• Correlation vs. causation
• Generalizing to a population that wasn’t included in the study

How are the data presented?

• Relative vs. absolute risk
• What are the axes on the figures?
• What statistics did they use? Are the data significant?

Looking for more resources? I discussed study designs quite a bit in my podcast titled “A Beginner’s Guide to Scientific Research.” For an example of how to identify limitations, see my article deconstructing an epidemiological red meat study.

Critical Analysis Is a Skill

Like any other skill, developing a critical eye for research papers takes time. Initially, you might need to read a paper a few times to fully understand it or look to other resources for additional background knowledge, but the more you practice, the easier it gets. Try to move more toward the second approach over time, as you become more comfortable with the literature, and be sure to experiment. I’ve presented the most common approaches here, but it’s more important that you find an approach that works best for you.

Learn from Others

Hearing other researchers’ interpretations of a study can be helpful, especially for topics that you are less familiar with. You can achieve this by starting a journal club at your clinic or institution, or joining the discussion online.

ResearchGate: A great option to read and discuss publications with other clinicians and researchers and access full-text articles. You must be a member of an institution to join.

Wiki Journal Club: A “collaborative website providing concise summaries of landmark clinical trials.” Wiki Journal Club attempts to present any limitations, criticisms, and issues with the study, along with its results, in an objective manner. Anyone can view without being a user, and any registered user can edit. You do not have to have an institutional affiliation.

Takeaways

1. Always read with a critical eye. Ask yourself questions as you go.
2. Note the study design and use this to think about potential limitations.
3. Once you become familiar with a topic, try to move towards the “results first” approach.
4. Practice! The more studies you read, the easier it will get to identify limitations and flaws.