Interpreting Risk Reduction in Probiotic & Prebiotic Clinical Trials
November 2017. By Prof. Michael Cabana MPH MD, Professor of Pediatrics, Epidemiology & Biostatistics and Chief, Division of General Pediatrics, University of California San Francisco.
Over the last few decades there has been a rapid acceleration in the number of published studies and clinical trials focused on probiotic and prebiotic interventions. One common result that is reported is the change in risk of a condition or outcome after taking a probiotic or prebiotic supplement. News articles and broadcasts commonly highlight claims in clinical trials (e.g., “this trial suggests a 33% reduction in X…). However, in a world where news is sometimes transmitted in 140 characters or less, much nuance from a proper clinical trial can be lost. When assessing claims of risk reduction, it is important to evaluate and interpret these results in their proper context. Here are a few tips.
What type of risk reduction is being reported?
When assessing the claims from a clinical trial, determine whether the claim is being presented as a relative risk reduction or an absolute risk reduction. Sometimes the report may describe the risk of the outcome or disease directly compared to the normal incidence of the disease (i.e., incidence seen in the control group). This is a report of an absolute risk reduction. For example, if the control group had a 15% frequency of disease X and the probiotic group had a 10% frequency of disease X, then the absolute risk reduction is 5% (15%-10%=5%). Sometimes the report may describe a relative risk reduction, which is the % change between the risk in the probiotic group compared to risk in the control group. If the control group had a 15% frequency of disease X and the probiotic group had a 10% frequency of disease X, then the probiotic reduced your relative risk by 33% ([15%-10%]/15% = 5%/15% = 33%).
Is the risk reduction clinically significant?
If you notice that a relative risk reduction is being reported as statistically significant, you then need to ask yourself if the outcome is clinically significant. It is possible that a very large change in the relative risk reduction may not be clinically important. For example, if a probiotic intervention decreases the relative risk of disease X by 33%., this percentage sounds very impressive. However, if the baseline risk of contracting disease X is only 0.06% (e.g., it is a very rare condition), then the risk after the probiotic intervention is only 0.04% (still very rare, as reflected in the absolute risk reduction of 0.02%). Although the decrease of 33% that is reported as relative risk seems large, if you take into account the baseline risk, you realize that this is not clinically significant. The risk of 0.06% and 0.04% are essentially the same.
When evaluating an intervention, the context of the disease makes a difference. How often is this disease or condition occurring in the population being studied? The problem with reporting a relative risk reduction is that it is easy to overlook how common or uncommon the disease is to begin with.
Look for the “Number Needed to Treat”
One way to better assess the impact of an intervention is to calculate a “Number-Needed-to Treat” (NNT). The NNT is the inverse of the absolute risk reduction.
From our example above, a 33% relative risk reduction of a condition with a prevalence of 0.06% (e.g., a very rare condition), means that the probiotic intervention had an absolute risk reduction of 0.02%. The NNT would be equal to 1/[0.0002]= 5000. This NNT of 5000 means that you’d need to treat 5000 patients with the probiotic intervention to change the outcome of only one patient.
Take a different scenario. If the disease was much more common (e.g, 9% prevalence) and the relative risk reduction was still 33%, then absolute risk reduction would be 3%. The NNT in this case would be equal to 1/(0.03)=33.3. This NNT of 33.3 means that you’d need to treat only 33 patients with the probiotic intervention to change the outcome of one patient. This treatment is much more likely to be meaningful in the population.
The NNT is a quick way for clinicians to evaluate an intervention to take into account the risk reduction in the context of the baseline risk.
Conclusion
When examining the results from clinical trials, just looking at percentage changes can be deceiving. Unfortunately, relative risk reduction often results in more sensational headlines, so beware of how the press, and even top quality journals, report study results. When assessing the clinical trial results in the context of clinical care, keep in mind how common or rare the disease is. Even a large percentage change may not make a big difference overall in patient outcomes if the initial risk was very low to begin with. Evaluate and interpret clinical trial results in their proper context.
