Make The Scientific Method Work For You!

We've all experienced this before: Someone we know tells us about a new study saying that a certain supplement, style of training, or way of eating is either the best thing ever, or that it "causes" a horrible disease or injury. The specifics come and go; the one constant is the way people seem to buy wholesale into bad science and big misunderstandings.

So why does it happen? Because a bunch of media people misinterpret a study, or fail to understand when a study's completely flawed? Sometimes, but it's also because there's a perceived separation between scientists and the rest of the world. Science does its thing, and the rest of the world sits back and waits for the answers.

Enough! Now more than ever, we need transparency. I'm a scientist, and I want you to be one, too. We all need to know how to look more critically at grandiose-sounding research and claims, as well as how to be more scientific in how we pursue our health and physique goals. In both cases, this starts with deepening our understanding of the scientific method.

Before we can tackle specific cases, we need to understand the scientific method itself. This is the system that guides everything that happens in science, from the lab to application. I want to help you understand how you can make it work for you.

Step 1: Ask a question

This step is deceptively simple, because in research, just as in life, asking the right question makes all the difference. For example, a question such as "What is the best workout program?" isn't a good question for either a scientist or a trainee. The question is too broad, and for that reason it's likely to lead to many different and opposing answers depending on goals, individual experience level, and testing methodologies. Furthermore, "the best workout" depends on the context of the workout, the goals of the individual, the training status of the individual, and a whole host of other variables.

Asking the right question depends on your goals, experiences, and other training variables.

Here are some examples of questions you could ask, to help steer your training in a more scientific direction:

Is a periodized training program superior to a nonperiodized training program for increasing strength in resistance-trained adult males?
Is a high-protein diet superior to a normal-protein diet for maintaining muscle mass in resistance-trained adult males during caloric restriction?
Can creatine improve strength in resistance-trained athletes?

The questions above are good staring points because each can be tested, has defined variables, targets a defined population, and asks a specific question in the proper context.

Step 2: Do background research

Once we have a question, then it's our job to find out what data is out there currently. If you're a researcher, your answer may already exist in a different study. If you're just someone looking to take a more scientific approach to training, your research can help you establish the structure of your experiment.

So where do you look? While the internet is a great source of information, the obvious problem is sorting out good information from bad. For researchers, the hunt often starts at PubMed, a database that contains an archive of nearly every biomedical study done in the last 50 years. Many of them are only available as abstracts—short summaries of the experiment and findings—but in some cases you will be able to get free access to the study.

Of course, PubMed can also be a minefield for someone who doesn't know their way around studies. At first, rather than searching PubMed to decide what workout to do or what supplements to take, it's OK to cast your net a little wider. Try looking up the studies referenced in articles on sites like Bodybuilding.com whenever possible, to get familiar with reading abstracts and studies.

For researchers, the hunt often starts at PubMed, a database that contains an archive of nearly every biomedical study done in the last 50 years.

Even if you can't get the cut-and-dried answer you seek this way, some strategic searching can help you find more information about your question. For example, we may find some good studies to help us with our initial questions by searching the following terms:

Resistance training periodization strength
High protein diet muscle lean body mass resistance training caloric restriction
Creatine strength resistance training athletes

The more specific you can make your search terms, the better your chance of finding the research you want. If you simply search "creatine," you are going to get thousands of hits, the majority of which won't be relevant to your question.

For future reference, specific journals I recommend following for nutrition and exercise information are:

Step 3: Form a hypothesis

If you do specific background research and don't find a consensus answer to your question, it's time to look at developing an experiment. But first, you should develop a hypothesis, also known as an "educated guess." It's the idea your experiment should either end up supporting or disagreeing with.

One of the most common mistakes researchers like me point out about media coverage of science is how writers tend to draw conclusions that aren't in the hypothesis of the study they cite. A recent example was a study that concluded a high-protein diet leads to increased levels of the cellular growth factor IGF-1.

Previous studies had linked elevated IGF-1 levels to certain cancers, so plenty of writers took it upon themselves to imply or state outright that high-protein diets could cause cancer.¹ Those of us in the know could only shake our heads, because we knew other studies had actually shown a high-protein diet led to a reduction in tumor size when compared to a high-carb diet!²

The moral of the story: Keep your hypothesis focused, so you can be confident in the conclusions you draw from it.

Here are some possible hypotheses you could create and test for our three questions:

A periodized program will produce better strength gains compared to a nonperiodized resistance training program in trained people.
A high-protein diet will improve muscle retention while in a caloric deficit compared to a normal protein diet.
Creatine will improve strength in resistance-trained individuals.

Step 4: Test your hypothesis via experimentation

Once you've formed your hypothesis, the real fun can begin. It's time to develop an experiment to empirically test your question, and then begin that experiment!

During this step, it's critical that you construct an experiment that controls as many variables as possible. You want to be confident that the answer you get is due to the experiment itself, not some other variable.

For example, there have been a few studies that concluded ketogenic diets produce greater fat loss than mainstream diets. However, most of these diets did not control for protein intake, meaning the protein intake was greater in the ketogenic diet group. Since protein has a thermogenic effect, this could be a variable impacting fat loss.³

To Test For Bench-Press Strength, You Have To Control All The Other Variables As Best You Can.

Here are some examples of how you could inadvertently confuse your results in our three experiments:

You start a new job that is labor intensive at the same time you start the periodized program, and note that your strength decreases over a few workouts. You incorrectly conclude that periodization doesn't work, rather than noting that it's likely you have not yet adapted to the workload of your new job.
You increase your protein intake during a caloric deficit, but also start doing massive amounts of cardio, and notice that your muscle mass decreases. You incorrectly conclude that the high protein did not help protect your muscle mass, when it was the confounding variable of excess cardio that likely caused the muscle loss.
You don't track your calories or macros strictly, or you make dramatic changes to your training while taking creatine, and thus conclude creatine didn't work. Since you were inconsistent with diet and training, it's more likely that any modest benefit from creatine supplementation was washed out by the large fluctuations.

If you want results you can be confident of, you need to be faithful to your experiment all the way to the end! That said, the perfect study doesn't exist. There are always limitations, and the job of a good scientist is simply to minimize them as much as possible.

Step 5: Analyze the data and draw conclusions

This is where many researchers and science writers get it wrong. As much as it seems that data should be intuitive, you have to understand what story the data is telling you.

Here are some questions to hold your data accountable:

Does your data agree with your hypothesis?
If it does, why do you think so? If it doesn't, why do you think it doesn't?
Do you have outliers—statistical oddities—that you didn't predict?

Now it's your job to come up with a mechanism that explains why your results are what they are. This is not always easy to do.

For example, I remember reading a paper that concluded casein was superior to whey for anabolism.⁴ This seemed to contradict what I and other researchers had observed. Come to find out, these researchers measured whole-body protein synthesis and did not directly assess muscle protein synthesis.

Their conclusion, in my opinion, was inappropriate for the experimental test that was conducted. Studies directly assessing muscle protein synthesis consistently demonstrate whey to be superior to casein.⁵ But, once again, writers in the media took the conclusion and ran with it without reading the fine print.

It doesn't mean the aforementioned researchers were evil or had ill intentions. It simply means they came to an incorrect conclusion based on that particular data set using their particular experiment. We as researchers must be careful what conclusions we draw and understand the limitations of our research.

Here's how a good analysis could look for our three questions:

Overall, my strength improved, and I added 20 pounds to my squat over the course of an eight-week periodized program. However, during one of the sessions, I squatted significantly less than I normally would for that particular rep range. Even though I had one or two bad sessions, this was likely an outlier and is not indicative that the routine didn't work, since overall my strength improved.
I followed a high-protein diet, but I lost some muscle mass. However, it's more likely that the added cardio led to the reduced lean body mass, rather than the addition of extra protein to my diet.
My squat one-rep max (1RM) went up by 10 pounds, although I'm not sure how much to attribute to the creatine. However, my workout quality went up, recovery was improved, and I missed fewer reps than usual, all of which lead me to believe the creatine was helpful.

It should go without saying that keeping accurate and thorough notes is important for analyzing your data. This is where a workout and nutrition log is worth its weight in gold!

Step 6: Accept, refine, or reject your hypothesis

It's simple: Does the data agree with your initial hypothesis? If not, you must reject it and look for other explanations. If it does, you can tentatively accept it. Or if it's not so clear-cut, we may need to refine our hypothesis. This isn't the end; it's actually a new beginning. Every experiment brings more questions, leading us to form a new hypothesis and put it to the test.

Every experiment brings more questions, leading us to form a new hypothesis and put it to the test.

You can change a minor variable, like a dosage or how you grip the bar, or you could change a lot, such as transitioning to an entirely different program. Here's how you might wrap up your experiments with our three questions:

My data appears to support my hypothesis, as I improved my strength over the course of the program.
My data shows that I lost muscle mass following a high-protein diet; however this is confounded by the fact that I added in additional cardio, so I need to collect more data in order to come to a sound conclusion.
My data appears to support my hypothesis, because creatine was the only supplement I added, I kept my nutrition and training consistent, and I added 10 pounds to my squat.

Think Like a Scientist

Keep in mind that, in science, nothing can be proven. That's right; you can't prove anything. You can disprove something, and you can conclude that your data appears to support your hypothesis, but it's always possible that a variable you didn't control for was influencing your results.

That may sound skeptical, but that's kind of the point. Being more scientific is about using skepticism and objectivity as a tool to your advantage. What seems cut-and-dried never is—and that's a good thing! The only way to find the truth is to keep going back, refining your hypothesis, and starting over again as systematically as possible.

This is the ultimate journey! Join me on it and use the scientific method to build your best body.

References

Walton, A. (2014, March 4). Why High-Protein Diets May Be Linked To Cancer Risk. Retrieved from http://www.forbes.com/sites/alicegwalton/2014/03/04/the-protein-puzzle-meat-and-dairy-may-significantly-increase-cancer-risk/
Moulton, C. J., Valentine, R. J., Layman, D. K., Devkota, S., Singletary, K. W., Wallig, M. A., & Donovan, S. M. (2010). A high protein moderate carbohydrate diet fed at discrete meals reduces early progression of N-methyl-N-nitrosourea-induced breast tumorigenesis in rats. Nutrition & Metabolism, 7(1), 1.
Johnston, C. S., Tjonn, S. L., Swan, P. D., White, A., Hutchins, H., & Sears, B. (2006). Ketogenic low-carbohydrate diets have no metabolic advantage over nonketogenic low-carbohydrate diets. The American Journal of Clinical Nutrition, 83(5), 1055-1061.
Boirie, Y., Dangin, M., Gachon, P., Vasson, M. P., Maubois, J. L., & Beaufrère, B. (1997). Slow and fast dietary proteins differently modulate postprandial protein accretion. Proceedings of the National Academy of Sciences, 94(26), 14930-14935.
Burd, N. A., Yang, Y., Moore, D. R., Tang, J. E., Tarnopolsky, M. A., & Phillips, S. M. (2012). Greater stimulation of myofibrillar protein synthesis with ingestion of whey protein isolate v. micellar casein at rest and after resistance exercise in elderly men. British Journal of Nutrition, 108(06), 958-962.