I started asking yesterday about learning how to decipher science papers. I think that’s what you’ve honed from the beginning with the Healthy Skeptic view. I’m reading a lot of papers since ADAPT began. I need help in this area of discernment. I asked my sister and her husband, biology professors at an upstanding college, and they were not helpful or encouraging, unfortunately, about my ability to learn this skill. They say it takes a lot of work, deep knowledge of subjects, statistical analysis, experimental design, knowledge of bias, also to toss anything not peer-reviewed, and even then, major flaws can be missed. It sounds daunting and yet I’m determined with or without their encouragement. Any suggestions you’re going to teach us, or resources?

Dr. Amy Nett:  I’m kind of disappointed to hear that they weren’t too encouraging because I would say you can, of course, learn how to read scientific papers. You can do anything you put your mind to. It’s a skill and I think it’s a matter of practice. I know you’re asking Chris directly, so if you want to post this to Chris so that he also sees it, we can certainly pass it along, but I think, to some extent, it’s familiarity with the research and just getting used to scientific terms. I think it was last week someone asked, “How do I learn biostatistics?” And honestly, in medical school they teach us biostatistics and that’s where you sort of learn the basics of what you’re looking for when you’re reading studies, and some of it’s just obvious. Look at who’s funding the study. Was it the company who produced the drug who’s funding the research on the drug efficacy? Then that study is probably not going to be great. It is difficult to interpret the scientific literature, but I think the more you read the scientific papers, you get a sense for kind of what’s useful and what’s trying to hide things.

One thing you might consider doing … I don’t know if you’re familiar with Denise Minger, but she has a blog and she recently changed the name of the blog, so I apologize. Denise is absolutely brilliant when it comes to interpreting studies. On her blog posts, and these are really long posts, but they’re actually taking a study and saying, well, here’s why I’m not sure I agree with their conclusions, and she sort of breaks it down. Like, well, they looked at this, but they didn’t take this or that into account. I think she sort of rose in popularity when she actually did a huge critique of The China Study and she really went through all the data and said, I think the data for The China Study was cherry picked and here’s why and this is how I would go through it. I think that was an 18-page critique. Go through those, take the papers that she’s critiquing, take her actual critique, and look at those. I think she does a good job of sort of laying out some of her thoughts and why it works and why it doesn’t.

I think Chris also has done something similar with studies. I think there was one on meat. I don’t remember exactly, but Chris has done the same thing, where he takes some articles and he’ll say, well, you can interpret it this way, but this is why I wouldn’t interpret it that way.

If you can take those and then see how people are thinking through this, I think it will give you a framework of how to approach research articles. That’s one way to think about doing it, but honestly, I think it’s just picking up papers and reading them and getting more comfortable. I don’t know how deep you need to go into really understanding biostatistics, if that’s as important.

Then you’re saying that your sister and her husband said to look at peer-reviewed research. That’s something else that’s really hard to do, but look at the journals that these things were published in because some journals, we’ve said, are like pay-to-play journals, so you basically submit a study along with a fee and they accept it and publish it. That’s kind of useless because nobody’s actually reviewing it and it doesn’t meet standards. You can just go to reliable sources, so if you’re on PubMed, get familiar with some of the major scientific journals and stick with those for what you’re reading. But last week, one of the things I mentioned was that you can go to a community college and just take a course in biostats or something like that. There are definitely resources. Look at a basic intro book for biostatistics or an online course for biostatistics, but that might give you a better sense for different ways to design studies, some of the best ways or the most flawed ways that studies can be put together.

Those are some ideas in terms of getting started and thinking about the framework for how to do it, but it’s really just practice and repetition. I don’t think Chris is going to be doing a specific “how to interpret a scientific study” [How to Read and Understand Scientific Research] [RHR: A Beginner’s Guide to Scientific Research], but I think those are some resources to hopefully get you started, and Denise Minger’s blog post is fantastic, so that might be a fun read to go through that. She really lays it out, how to think about it.