Scientific Research And Scientific Uncertainty

Some say that every particle of matter has an opposite, a piece of antimatter. The same can be said for many scientific studies; for every study that says X is true, there is another study that says Y is true and X is either wrong or was not as important as everyone thought. Controversy in science is good and keeps the research interesting, presenting new challenges that force our knowledge to grow. The public, however, as a rule is uneasy with the concept that uncertainty is a fact of scientific life.

 

Another important lesson is that no study is perfect. A decent statistician can find a flaw in almost any study. Many times researchers must make difficult decisions that can strengthen one part of a study at the expense of another. One problem we commonly see in research on nutritional supplements is that many early studies attempted to investigate the short-term effects of supplementation, an approach that often yields discouraging results. We now understand, however, that diet and supplements achieve their greatest benefit over months or years, not just a few days. Healthy living is a lifelong endeavor, not something to practice for a couple of weeks and discard once we feel or look better. As for asthma, we now know that healthy living works its magic not by aborting acute attacks but by preventing future attacks, something difficult to demonstrate in a research study.

 

There is also the whole subject of the relative strengths and weaknesses of different study designs, a subject that could easily fill several books. For our purposes, all research studies quoted in this book are subject to the same exacting standards and criticisms as any other type of scientific endeavor and display the same strengths and weaknesses inherent to all methods of inquiry.

 

Now, don't get me wrong, physicians do know a lot about the human body. Physicians know a lot about asthma and how to treat it, and few doctors would deny that allergies and asthma often go hand-in-hand. However, controversy rages on over how much allergies contribute to asthma. So, while there are many facts about asthma that most physicians agree on, there are many more unresolved questions that are subject to intense debate.

 

We also have to understand that scientists and physicians are trained to remain skeptical, to never accept anything at face value. For instance, it's not good enough that one study reports that vitamin C helps asthmatics. The "scientific method" demands that additional studies be performed to see if similar results can be achieved, a quality known as reproducibility. Reproducibility is central to the scientific method and is a major reason why controversy exists in medicine. It is also the reason why it often takes years for a particular finding to be accepted by the scientific community. Reproducibility is so important that, as far as the scientific method is concerned, there can be no facts without it.

 

Scientific discovery is not like a movie, with the wild-haired scientist surrounded by test tubes suddenly shouting, "Eureka!" Finding the truth is hard work and normally involves years of research coupled with persistent controversy. How we get to know what we know can be divided into three general stages: hypothesis, research-publication-controversy, and acceptance-rejection. This process of discovery can last decades, but it is the only way we can be remotely certain about the accuracy of scientific research.

 

What usually happens first is that someone reports an expected or unexpected finding in a study. From this new finding, a hypothesis is generated. For instance, a group of researchers are studying the relationship between diet and health and find that people with diets rich in fresh fruits have a lower risk of asthma. Another scientist reads their report and says to herself, "That's interesting. I know that fruits are rich in vitamin C and that vitamin C is a powerful antioxidant. I also know that oxidative stress plays a role in asthma. I wonder if giving asthmatics extra vitamin C will reduce their symptoms?" This is the hypothesis: Can vitamin C supplementation reduce asthma symptoms?

 

The researcher then designs a study to test her hypothesis. She develops a randomly controlled, double-blind study by giving half a group of asthmatics vitamin C and the other half a placebo. She then follows these two groups for a period of time, during which she measures several asthma-related variables like lung function and asthma severity scores. Once the study has been completed, the test results are compared between the two groups and presented to the scientific community, usually by being published in a scientific journal.

 

Other scientists see her report and decide to reproduce those results to determine if they're valid. This is where things get complicated. How a study is designed depends on the researcher and the type of data being collected. There are several basic study designs, each with their own strengths and weaknesses, and no two studies are exactly alike. As a result of these efforts, multiple studies examining the relationship between asthma and vitamin C begin to appear in the scientific literature. What often happens in early research is that some studies say vitamin C helps, some say vitamin C doesn't help, while other studies are inconclusive.

 

Researchers then critically review the published studies for flaws and design improved studies that they hope will yield reproducible results, a process that can take years to decades. This cycle of publication followed by critical review and additional publications is called the research-publication-controversy stage.

 

Ideally, over a period of years, what you find is a gradually evolving body of literature with more consistent findings. Once this occurs, we are at the acceptance-rejection part of the process. Yet, no matter how consistent the findings are among a group of papers, it is not uncommon to see newly published work that contradicts the established consensus. Sometimes these contradictory studies simply result from design flaws and are quickly dismissed. Other times, such studies represent the first step in toppling a medical dogma.