The Fundamental Problem with Most Nutrition Research

Published on May 2, 2018

Science Based

Nutrition research relies heavily on diet recall and food diaries. But how accurately do people report what they eat? Read on to learn about the flaws of diet reporting, the implications for interpreting nutrition research, and the future of the field.

What did you eat for lunch two days ago? How many servings of red meat per week did you consume on average for the past month? If you are having trouble with those queries, you’re not alone. Yet questions like these are at the heart of nutrition research, and dietary guidelines are based on the answers. This article will cover how inaccurate diet records, especially underreporting, plague nutrition studies and how we can’t trust the conclusions that stem from faulty data.

The History of Nutrition Research: The Outdated Reductionist Approach

Nutrition research first emerged around 1932, when vitamin C was shown to cure scurvy (1). From here, a “reductionist approach” developed. Basically, researchers identified nutrients that, when deficient, caused disease. Thiamine deficiency leads to beriberi, vitamin D deficiency leads to rickets, and so on. Next, the necessary intake of that nutrient to prevent said disease was determined.

Whether done consciously or unconsciously, underreporting plagues observational nutrition studies.

For awhile, this approach helped prevent and cure many acute diseases. But this model doesn’t work so well for chronic diseases such as obesity, heart disease, and osteoporosis because the causes are more complex than a single missing nutrient, and these conditions develop over a lifetime. Similar to my belief that conventional medicine misses the mark for treating chronic disease, conventional nutrition research isn’t getting to the root causes, either.

Dietary Intake Assessments Make Nutrition Research a Pseudoscience

Nevertheless, today’s nutrition research still attempts to pinpoint one nutrient, or one food type, and relate it to disease. Take a look at the titles of these peer-reviewed articles:

Nutrition studies are either randomized controlled trials or observational studies. Most nutrition research is based on observational studies, which can be one of three types:

Cross-sectional studies look at the present health state of people and relate it to current eating habits
Prospective studies choose a study population in the present and track their food, exposures, and health outcomes over a period of time
Retrospective studies observe the present health state of people and relate it to records of what they ate in the past

In short, researchers observe what people eat/ate and relate it to what diseases or health issues they get/have.

Methods of food reporting include 24-hour dietary recall interviews, food-frequency questionnaires, and seven-day dietary recall interviews. Keeping track of or recalling what you eat can be inconvenient and imprecise. Can you remember how many times this past month you consumed a serving of green vegetables? I think you see my point. Herein lies the biggest limitation of observational nutrition studies: the inaccuracy of diet reporting. People underreport, exaggerate, forget, and even consciously omit what they eat.

Research has identified common reasons why food diaries and recalls can be so inaccurate: (2, 3)

People tend to underreport foods socially considered “bad,” like red meat and alcohol
People overreport foods socially considered “good,” such as vegetables and fruits
People may not know all the ingredients in restaurant or prepared foods
People don’t weigh or otherwise measure portion sizes
People find tracking every bite and meal inconvenient
People are human and just can’t remember every little thing they eat

Despite the shortcomings of the data, dietary advice and public policy rely almost solely on what people say they eat! As a result, nutritional studies have been met with heavy criticism. A recent article from the Mayo Clinic Proceedings claims that because nutrition studies “cannot be reliably, accurately, and independently observed, quantified, and confirmed or refuted,” they do not follow the scientific method and should be regarded as “pseudoscience” at best (4). I would have to agree.

Underreporting Is Ubiquitous

Whether done consciously or unconsciously, underreporting plagues observational nutrition studies. In general, women and obese people underreport most often.

The NHANES program has published surveys since the 1960s to assess health and nutrition of people based on personal diet reports and physical examinations. A ratio of reported energy intake to basal metabolic rate below 1.35 is considered “implausible” and indicative of underreporting. In the nine surveys, when men and women were analyzed separately to create 18 groups, only three of the 18 groups’ diet reports were physiologically plausible (5). In kcal, the mean underreporting was 281 kcal/day for men and 365 kcal/day for women. In obese individuals, underreporting rose to 717 kcal/day for men and 856 kcal/day for women.

The EPIC study contains the largest set of data on people’s lifestyles, diet, and biomarkers that can be correlated to cancer and cause-specific mortality (6). However, underreporting was found across the board, with around 10 to 14 percent of participants identified as “extreme underreporters” (7).

In the OPEN study, researchers compared 24-hour recalls (24HR) and food-frequency questionnaires (FFQ) to doubly labeled water and urinary nitrogen, biomarkers for energy and protein intake, respectively. Both FFQ and 24HR were underreported compared to the biomarkers, with FFQ even lower than 24HR. Obese individuals tended to underreport more than non-obese. (8)

Another study followed 30 obese men for a week, comparing food diaries to doubly labeled water measures (9). Average underreporting was 37 percent, with selective underreporting of fat.

Occasionally, researchers attempt to identify and remove underreporters’ data. One study recommended using plus or minus one standard deviation as a cut-off when comparing reported energy intake with predicted total energy expenditure (10). In other scientific fields, researchers need compelling reasons to remove outliers, and the cut-offs are generally much more stringent, such as 2.5 or three standard deviations away from the mean.

Weeding out underreporters introduces new challenges. What if those who underreport are the unhealthiest, the ones we need to study the most? Because obese individuals are more likely to underreport, relying on their food diaries to study obesity will lead to false conclusions. For example, imagine that a study group underreports fat intake, and the data show that high-fat diets correlate with obesity and heart disease. The reported “negative effects” of a high-fat diet in this case would be overestimated, if the correlation even exists at all (11).

We wouldn’t publish data and develop recommendations based on research using a scale that measures weight inaccurately, yet that’s exactly what still happens in nutrition research. A 2015 article hits the nail on the head about using NHANES data to form national dietary guidelines:

“Therefore, these protocols and the resultant data should not be used to inform national dietary guidelines or public health policy, and the continued funding of these methods constitutes an unscientific and major misuse of research resources” (4).

New Tools for Measuring Food Intake

In lieu of traditional food-frequency questionnaires and 24-hour diet recalls, nutrition research is slowly shifting toward newer technologies for measuring food intake, including the following:

Thermodynamic models (12, 13)
Remote swallowing sensing devices (14)
Remote food photography for more accurate portion sizes and better memory (15)
Personal electronic devices to easily enter food at any time
Doubly labeled water to measure energy intake and urinary nitrogen to measure protein intake

There Is No One-Size-Fits-All Diet

If the average person tries to determine what to eat from nutrition research, he will come to the conclusion that most foods cause cancer! One interesting study looked at 50 common ingredients from random recipes and found that 80 percent were correlated with cancer risk in the literature (16).

In light of flawed nutrition research and conflicting studies published constantly, most patients don’t know what to eat. This is one reason why I view the Paleo diet as more of a template and work individually with patients—because there is no one-size-fits-all diet. Some patients cannot tolerate dairy; others do best on the GAPS diet; some patients need to monitor their cruciferous vegetable intake. My general recommendations include:

Focus on quality, not quantity. Healthy hunter–gatherer civilizations have widely varying diets, in terms of individual components and macronutrient percentages. For instance, Kitavans consume over 60 percent of their daily energy as carbohydrates (17), while the Inuit consume almost 60 percent of their daily energy as fat (18).
Abandon processed, packaged foods. Packed with refined sugars and rancid industrial seed oils, they are inflammation in a box.
Support food practices that are sustainable and humane. CAFO meats and toxic pesticides are neither.