Research Studies: Why the Media So Often Gets Them Wrong

Red meat on the chopping block again? While epidemiological research is useful for identifying potential associations between dietary and lifestyle factors and various health outcomes, it also has some major limitations. Read on to learn about the perils of observational epidemiology in this case study about red meat and diverticulitis.

---

In early 2017, a manuscript titled “Meat intake and risk of diverticulitis among men” was published in the journal Gut (1). While the authors of the publication spoke cautiously of an association identified between red meat and diverticulitis, an inflammatory disease of the colon, the media and health bloggers published sensational headlines such as:

“Eating red meat can cause diverticulitis” (2)

“Diverticulitis symptoms? Red meat diet can cause condition often mistaken for bowel cancer” (3)

“Diverticulitis study: More bad news for lovers of red meat” (4)

“New study links red meat to 58% higher chance of developing painful diverticulitis” (5)

This is far too common. As a clinician, it’s important to be able to wade through catchy media headlines and critically analyze the primary literature. In this article, I’ll demonstrate the perils of observational epidemiology using this new red meat study and provide some tips on how to critically review a study.

Back to Basics: Research Design

Before we dive into the study results, let’s review some basics. Epidemiology is the study of the patterns, causes, and effects on health and disease conditions in well-defined populations. An observational study is one that draws inferences about the effect of an exposure or intervention on a population of subjects. The researcher has no control over the subjects and is simply observing a population of people and drawing inferences about the effects of a diet or lifestyle factor on various health outcomes.

There are a few types of observational studies used in epidemiology:

1. Cross-sectional studies take a snapshot of the present and seek to correlate patterns in the present with current states of health or disease.
2. Retrospective cohort studies typically look at disease in the present and review past records to determine what prior exposures might be associated with health or disease.
3. Prospective cohort studies identify a study population in the present and track both their exposures and their health outcomes going forward over a follow-up period.

Causation and the Hill Criteria

In general, observational epidemiology cannot infer causation. However, a group of guidelines known as the Hill criteria for causation were established in 1965 by the epidemiologist Sir Austin Bradford Hill. These criteria provide a framework for determining the strength of observational evidence and help determine if an epidemiological association is likely to be causal (6). The criteria are:

1. Strength of association: the larger the effect, the more likely it is causal
2. Consistency: if multiple studies with multiple different sample populations find the same association, it is more likely to be causal
3. Specificity: if a specific exposure causes a specific disease and there is no other likely explanation, there is a greater chance that the relationship is causal
4. Temporality: the exposure must precede the effect in order to potentially be causal
5. Biological gradient: if a dose response is observed (i.e., greater exposure leads to greater effect), the relationship is more likely to be causal
6. Plausibility: if a plausible mechanism can be provided based on current knowledge, the association is more likely to be causal
7. Coherence: agreement between laboratory and epidemiological studies increases the potential of causality
8. Experiment: if an experimental intervention occurs and the disease ceases when exposure ceases, the relationship is more likely to be causal
9. Analogy: if a similar factor is already known to cause the disease, it is more likely that the factor in question is also causally associated with the disease

With all of this in mind, let’s move on to the red meat–diverticulitis study.

What Was the Study Design?

The Health Professionals Follow-up Study (HPFS), a large prospective cohort study of male health professionals, was used in this analysis of meat intake and diverticulitis. The men were aged 40–75 at enrollment in 1986 and were followed for 26 years to track their health outcomes. Every four years, the men were asked to fill out food frequency questionnaires, which asked the participants detailed questions about how often they consumed foods (including different types of meat) of a standard portion size during the past year. Provided multiple choice responses ranged from “never or less than once per month” to “six or more times per day” (1).

Should you trust the latest health headline? The trouble with observational studies

What Did They Find?

Of 46,461 men included in the study, there were 764 incident diverticulosis cases over 26 years of follow-up. (Because some men died before the full 26 years, the authors report the data in person-years; the risk is therefore reported as the chance of a participant developing diverticulitis in any given follow-up year.) Their two main findings are summarized below:

“Overall, total red meat intake was associated with an increased risk of diverticulitis.”

 “The observed link between total red meat intake and risk of diverticulitis appeared primarily driven by consumption of unprocessed red meat.” (1)

Many clinicians and scientists would take these conclusions at face value and recommend reducing meat intake. However, there are several limitations to this study that are common among nutritional epidemiology studies and should be considered when interpreting the results. We’ll discuss each in turn.

Limitation #1: Self-Reporting Bias

In order to assess the influence of dietary factors on disease outcome, you need to be able to quantify dietary intake. One of the most common methods is the food frequency questionnaire (FFQ), which asks participants about their average intake of various foods over a given time period. However, there are inherent issues with self-reporting, and despite their widespread use, FFQs have received quite a bit of criticism (7, 8). In fact, dietary intake assessment has been rumored as the “weakest link” in nutritional epidemiology.

Diverticulitis outcomes were also self-reported. This is likely less of an issue in this particular study since the study population was healthcare professionals, who are more likely to be familiar with disease states and medical terminology.

Limitation #2: The (Un)healthy User Bias

I have discussed the healthy user bias before in my work and on my podcast. The basic premise is that if popular wisdom says “red meat is bad for you,” people who are committed to their health will tend to eat less red meat. These people (“healthy users”) are also more likely to engage in other health-promoting behaviors, such as exercise, not smoking, and eating more fruits and vegetables. People who engage in unhealthy behaviors like smoking (“unhealthy users”) are more likely to also eat red meat. Because of this, it becomes difficult to separate eating red meat from simply caring about your health.

Indeed, the authors write: “Compared with men with lower intake of red meat, men with higher red meat consumption smoked more, used NSAIDs and acetaminophen more, and were less likely to exercise vigorously.” Fiber intake among these men was also lower.

The researchers did factor out different variables (BMI, physical activity, smoking, fiber intake, and several over-the-counter medications) and still found a negative effect of red meat. However, there could potentially be other confounding variables underlying a healthy user bias. As the authors state: “The possibility of residual confounding can never be ruled out” (1).

Limitation #3: Study Population

Epidemiological studies are always performed in a defined population, and it’s important to interpret any findings in the context of this population. In this study, all of the participants were male healthcare professionals aged 40–75 at enrollment in 1986. This would make them 66–101 years old at the conclusion of follow-up in 2012. We should therefore be careful about generalizing this information to adults, women, and individuals who are not healthcare professionals.

As with most modern nutrition studies, the average participant here was also likely eating a Standard American Diet high in refined carbohydrates and processed foods. We might ask: is it really the beef burger? Or is it the wheat bun, processed fries, industrial seed frying oil, or high-fructose corn syrup ketchup? Would the association still exist in a group consuming a nutrient-dense, evolutionarily appropriate diet?

Limitation #4: Relative vs. Absolute Risk

Now, this is the real kicker. The authors only report relative risk:

“Compared with men in the lowest quintile (Q1) of total red meat consumption, men in the highest quintile (Q5) had a multivariable RR of 1.58 […] after adjustment for fiber and all other potential confounding variables” (1).

On first glance, this sounds like quite an impressive finding. Meat increases your risk of diverticulitis by a whopping 58 percent, right?

Not so fast. If you take a minute to comb through the data and calculate the absolute risk:

• The overall chance of developing diverticulitis in a given year, regardless of any factors:
  764 total cases / 651,970 total person–years of follow-up = 0.12 percent
• The chance of developing diverticulitis in a given year if you were in Q1 of red meat consumption (median 1.5 servings per week):
  106 cases / 134,819 person–years of follow-up = 0.08 percent
• The chance of developing diverticulitis in a given year if you were in Q5 of red meat consumption (median 12.4 servings per week):
  183 cases / 131,982 person–years of follow-up = 0.14 percent

When you divide 0.14 / 0.08 = 1.76, you can see that there is indeed a 76 percent increased relative risk of diverticulitis for those in Q5 versus those in Q1 of red meat consumption (this drops to 58 percent when you account for fiber intake and other variables). But, red meat consumption only increased the absolute risk of developing diverticulitis in a given year by 0.06 percent. Seems much less impressive, doesn’t it?

Reporting only relative risk is common in nutritional epidemiology and can be very misleading, especially in the cases of fairly uncommon diseases like diverticulitis.

So, Should We Be Concerned about Red Meat?

While I highlighted the limitations here, the study does have several strengths, including a large sample size and a separate analysis for processed and unprocessed meats. It was also a prospective cohort study, which is slightly more reliable than other observational designs, such as the cross-sectional study. The authors also attempted to control for the healthy user bias and the effect of age by performing multivariate and age-adjusted analyses.

In terms of the Hill criteria for causation, the strongest criterion supporting this study might be plausibility. When we find an association, it’s important to ask: is there a plausible mechanism? The authors note that red meat consumption in the context of gut dysbiosis can result in increased production of TMAO, a metabolite that has been associated with cardiovascular disease and inflammation (9). However, as I’ve written before, there is little evidence to suggest that red meat is the causative agent; rather, the evidence points to the disrupted gut microbiota as the real cause of inflammation.

Other Hill criteria that might support causation are consistency and a biological gradient. We’ve sure seen plenty of studies that find an association between red meat and bowel diseases, and some of these have found a dose response. Still, it’s important to note that all of these studies are performed in individuals eating a Standard American Diet and inevitably confounded by the same healthy user bias. Most of these studies also found only a modest association.

So, is it possible that red meat increases risk of diverticulitis? Yes. However, considering the confounding biases, study population, and small change in absolute risk, I’m not convinced that we need to be concerned about red meat intake in people eating a healthy diet.

Takeaways

1. Be wary of sensationalist media headlines. Always look at the primary literature yourself and read with a critical eye.
2. The results of a study should always be discussed in light of their limitations. This is true of all study designs. If you’re not familiar with the various study designs and their limitations, check out a beginner’s guide to scientific research on my podcast.
3. Correlation does NOT equal causation. Observational studies are valuable for identifying associations between diet and lifestyle factors and disease (and the Hill criteria outlined above can help evaluate the likelihood of causation), but only randomized controlled trials can confirm that a relationship is causal.