Misleading press about probiotics: ISAPP responses

By Mary Ellen Sanders, PhD, Executive Science Officer, ISAPP

It seems over the last couple of years, open season on “probiotics” has been declared. Responding in a scientifically accurate fashion to misleading coverage, whether it is in reputable scientific journals or in the lay media, takes time and care.

I want to be clear: well-conducted clinical trials, regardless of the outcomes, are welcome contributions to the body of evidence. No one expects that every probiotic will work for every indication. Null trials document this – they tell researchers to look elsewhere for solutions. Further, we must acknowledge the limitations and weaknesses of available evidence; unfortunately, not all trials are well-conducted. We also need to be just as diligent in criticizing press that is overly positive about probiotic benefits, which are not backed by evidence.

However, articles with misleading information are all-too-frequently published. Below are ISAPP responses to some of these stories.

  1. A paper on rhamnosus GG bacteremia in ICU patients led to headlines about ‘deadly infections’ and probiotic administration ‘backfiring’, even though no patients died and clinical outcomes were not collected. ISAPP responded to clarify appropriate context for understanding the safety issues raised from this paper. See Lactobacillus bacteremia in critically ill patients does not raise questions about safety for general consumers.
  2. The Wall Street Journal published an article condemning probiotics for reducing fecal microbial diversity. ISAPP responded with a blog Those probiotics may actually be helping, not hurting, pointing out the errors in the author’s thinking (equating diversity with gut health).
  3. A pair of well-conducted clinical trials that did not show impact of probiotics on pediatric acute diarrhea led to some ignoring all previous evidence and concluding that no probiotics were useful for acute pediatric diarrhea. ISAPP responded about the importance of putting new evidence in the context of the totality of evidence: L. rhamnosus GG for treatment of acute pediatric diarrhea: the totality of current evidence. Also, Dr. Eamonn Quigley, an ISAPP board member, published an independent response.
  4. Pieter Cohen concluded that evidence for probiotic safety is insufficient in an article in JAMA Internal Medicine. ISAPP’s response was published in a letter to the editor, along with Cohen’s response to our letter.
  5. Responding to two papers in Cell (here and here), and accompanying media coverage that called into question probiotic safety and efficacy, ISAPP published a detailed post Clinical evidence and not microbiota outcomes drive value of probiotics objecting to conclusions, and released a public statement.
  6. Jennifer Abbasi wrote a critical article about probiotics with the inflammatory title “Are Probiotics Money Down the Toilet? Or Worse?” ISAPP responded with the following blog post: Probiotics: Money Well-Spent For Some Indications.
  7. When Rao, et al incriminated probiotics as a cause of D-lactic acidosis, ISAPP posted a blog and published a letter to the editor of Clin Transl Gastroenterol objecting to this conclusion.
  8. ISAPP responded to a paper claiming that probiotics were unsafe in children: Probiotics and D-lactic acid acidosis in children and Brain Fogginess and D-Lactic Acidosis: Probiotics Are Not the Cause.

Board member and Professor Colin Hill wrote a blog post called Another day, another negative headline about probiotics? His post provides some useful questions to consider when reacting to a publication:

  • Is the article describing an original piece of research and was it published in a reputable, peer-reviewed journal?
  • What evidence is there that the strain or strain mix in question is actually a probiotic? Does it fit the very clear probiotic definition?
  • Was the study a registered human trial? How many subjects were involved? Was it blinded and conducted to a high standard?
  • What evidence was presented of the dose administered and was the strain still viable at the time of administration?
  • Were the end points of the study clear and measurable? Are they biologically or clinically significant to the subjects?
  • Did the authors actually use the words contained in the headline? “Useless”, or “waste of money”, etc?


Effects of the food matrix on probiotic’s efficacy: how much should we care?

By Gabriel Vinderola PhD, Researcher at the Dairy Products Institute (National Scientific and Technical Research Council – CONICET) and Associate Professor at the Food Technology and Biotechnology Department, Faculty of Chemical Engineering, National University of Litoral, Santa Fe, Argentina.

The issue of to what extent food components may affect probiotic efficacy when compared to the strain delivered as supplement has lately been the subject of debate. This is especially so in the context of the Codex Alimentarius guidelines on probiotics, presently under development.

When considering the importance of the food formulation delivering the probiotic, it’s worthwhile to keep in mind that people may get their daily probiotic together with an enormous variety of foods. For instance, one person may get the probiotic at breakfast along with a yoghurt or with cereal, whereas another person may choose to consumer a fruit juice, while a third may get the probiotic dose before a meal consisting of pasta, meat and vegetables. In those cases, the same strain can undergo gastrointestinal passage in the context of very different food exposures. Does this suggest that perhaps the specific food format is not so critical? What does research tell us?

An interesting, however in vitro, study was conducted by Grześkowiak et al. (2011). In this work, Lactobacillus rhamnosus GG was recovered from more than 12 foods and supplements and its ability to inhibit food pathogens was assessed in vitro. Authors showed that even when the inhibitory capacity was quantitatively different among isolates, the qualitative probiotic capacity of inhibiting pathogens was present in all of them. That is to say, the probiotic capacity had been retained to a somewhat greater or lesser degree, regardless the matrix.

Few human studies have measured to what extent a health endpoint changes when a probiotic is delivered in different food matrixes. For instance, Saxelin et al. (2010) showed that the administration matrix (capsules, yogurt or cheese) did not influence the faecal quantity of lactobacilli, but affected faecal counts of propionibacteria and bifidobacteria. However no health endpoint was considered in this study. Several studies demonstrate that dairy products are able to confer enhanced protection during gastrointestinal transit in in vitro settings (Vinderola et al., 2000; Sagheddu et al., 2018; da Cruz Rodrigues et al., 2019), suggesting that dairy products may be better at delivering an efficacious dose of probiotic. But again, no clinical endpoint was measured in these studies.

The first comparative study on the probiotic capacity of a strain delivered in food or supplement was reported by Isolauri et al. (1991). Authors demonstrated that Lactobacillus GG either in fermented milk or freeze-dried powder was effective in shortening the course of acute diarrhea. Later on, Meng et al. (2016) found similar patterns of immune stimulation when studying the impact of Bifidobacterium animalis subsp. lactis BB12 administration in yoghurt or capsules on the upper respiratory tract of healthy adults.

As these kinds of studies are scarce, we can look to meta-analysis where the same strain is compared for the same clinical endpoint, but in studies conducted by different groups in different matrixes. For instance, Szajewska et al. (2013) concluded that Lactobacillus GG delivered in capsules or fermented milk significantly reduced the duration of diarrhea and Urbańska et al. (2016) reported that L. reuteri DSM 17938 delivered in either capsules or infant formula reduced the duration of diarrhoea and increased the chance of cure.

In vitro studies find that survival of the probiotic delivered in different food matrices through a (simulated) gastrointestinal transit may quantitatively differ, but no matrix completely eliminates probiotic capacity. Human clinical trials comparing different matrices with a clear health endpoint are scarce, but a general conclusion seems to emerge: regardless of the food matrix, the probiotic effect is achieved.  When the data are assessed through meta-analysis, the top of the “levels of evidence” in the pyramid of evidence-based studies, the probiotic capacity exists for the same strain among different studies, conducted by different research groups, using different food matrices.

In many countries regulators require that the probiotic effect be demonstrated in the same food or supplement that will be offered to consumers. This is a conservative approach in the lack of other evidence, but it may be challenging at the same time for probiotic food development, as any new food, even similar to one already existing, may require new human clinical studies to demonstrate efficacy. This approach may raise economic and ethical concerns too, and be discouraging for the future of probiotics.

Surely additional clinical trials directly comparing effects among different delivery matrices would provide clarity on the importance of this factor to probiotic functionality. Until that time, regulators should enable probiotic food manufacturers to offer a sound scientific rationale that bio-equivalency of different matrices could be expected, and thereby circumvent the requirement need to re-conduct human clinical trials on probiotics delivered in new matrices.



da Cruz Rodrigues VC, Salvino da Silva LG, Moreira Simabuco, F, Venema K, Costa Antunes AE. Survival, metabolic status and cellular morphology of probiotics in dairy products and dietary supplement after simulated digestion. J Funct. Foods, 2019, 55, 126-134.

Grześkowiak Ł, Isolauri E, Salminen S, Gueimonde M. Manufacturing process influences properties of probiotic bacteria. Br J Nutr. 2011, 105(6):887-94.

Isolauri E, Juntunen M, Rautanen T, Sillanaukee P, Koivula T. A human Lactobacillus strain (Lactobacillus casei sp strain GG) promotes recovery from acute diarrhea in children.

Meng H, Lee Y, Ba Z, Peng J, Lin J, Boyer AS, Fleming JA, Furumoto EJ, Roberts RF, Kris-Etherton PM, Rogers CJ. Consumption of Bifidobacterium animalis subsp. lactis BB-12 impacts upper respiratory tract infection and the function of NK and T cells in healthy adults. Mol Nutr Food Res. 2016, 60(5):1161-71.

Pediatrics. 1991 , 88(1):90-7.

Sagheddu V, Elli M, Biolchi C, Lucido J, Morelli L. Impact of mode of assumption and food matrix on probiotic viability. J Food Microbiol. 2018, 2.

Saxelin M, Lassig A, Karjalainen H, Tynkkynen S, Surakka A, Vapaatalo H, Järvenpää S, Korpela R, Mutanen M, Hatakka K. Persistence of probiotic strains in the gastrointestinal tract when administered as capsules, yoghurt, or cheese. Int J Food Microbiol. 2010, 144(2): 293-300.

Szajewska H, Skórka A, Ruszczyński M, Gieruszczak-Białek D. Meta-analysis: Lactobacillus GG for treating acute gastroenteritis in children-updated analysis of randomised controlled trials. Aliment Pharmacol Ther. 2013 Sep;38(5):467-76.

Urbańska M, Gieruszczak-Białek D, Szajewska H. Systematic review with meta-analysis: Lactobacillus reuteri DSM 17938 for diarrhoeal diseases in children. Aliment Pharmacol Ther. 2016, 43(10):1025-34.

Vinderola G, Prosello W, Ghiberto D, Reinheimer J. Viability of  probiotic- (Bifidobacterium, Lactobacillus acidophilus and Lactobacillus casei) and non probiotic microflora in Argentinian Fresco Cheese (2000). J Dairy Sci. 2000, 83 (9), 1905-1911.


Efficacy and Effectiveness Studies

By Michael D. Cabana, MD, MPH

In the world of clinical trials, reproducibility (or consistency) of results across different clinical trials improves clinicians’ confidence in an intervention (Hill, 1965).  However, when reviewing the evidence for a probiotic or prebiotic supplement, the results are sometimes conflicting.  One study claims an intervention may work.  Another study claims that an intervention may not work. So how does the clinician deal with this situation?

To know how much confidence to place in any claim of benefit, clinicians need to consider the totality of the evidence and the quality of the studies. One tool is the systematic review process, which in an unbiased manner searches for all studies for a particular intervention, and when possible, combines results into a meta-analysis. The ‘summary’ of these data point to either an effect or no effect. The best way to combine data is using an individual patient-data meta-analysis (IPDMA). In addition, a clinician should determine whether the clinical trial is an effectiveness study or an efficacy study (Singal 2014).


Efficacy or Effectiveness?   

Efficacy studies ask, “does the intervention work in a defined (usually an “ideal”) setting?”  In general, the inclusion criteria for study participants will be very selective.  Patient adherence tends to be closely monitored. The clinicians conducting the trial may be specially trained in the intervention and its application. The intervention occurs in an ideal setting and the risk of other confounding interventions (e.g., unusual diets, concurrent treatments) will be limited.

On the other hand, effectiveness studies ask, “Does the intervention work in a real-world setting?”  The inclusion criteria for study participants tends to be less selective.  Patient adherence to the protocol is not necessarily strictly enforced. The clinicians conducting the trial tend to be representative of the typical physicians who would treat this condition.  The intervention occurs in a more ‘real-world’ setting where the presence of other confounding factors may be present.

For example, two relatively recent studies both examined the effect of a probiotic intervention, L. reuteri DSM 17938 for the treatment infant colic.  A study conducted by a team in Italy (Savino et al. 2010) noted that the intervention reduced colic symptoms; however, the study conducted by a team in Australia (Sung et al. 2014) showed no effect on colic.

Why the different results? In the Italian study, all the infants were breastfed.  In addition, the breastfeeding mothers limited their dairy intake.  The infants tended to be younger (mean age 4.4 weeks) and tended not to have other treatments for colic or gastrointestinal symptoms.  In contrast, the infants in the Australian study were breastfed or formula fed. The infants were older (median age 7.4 weeks) and were more likely to have been exposed to other treatment for gastrointestinal symptoms (such as histamine-2 blocker or proton pump inhibitors).  The infants were recruited from many different settings such as the emergency department.

Although both the Italian and the Australian study evaluated the same probiotic intervention for the same condition, the studies offer different information in terms of efficacy and effectiveness.  Describing a study as either an “efficacy” study or an “effectiveness” study is not always dichotomous.  Rather, these studies exist on a spectrum, from being more like an efficacy study versus more like an effectiveness study. In the example above, the Italian study had stricter criteria and fewer confounding factors.  As a result, it would tend to be classified as an efficacy study.  The Australian study enrolled infants with colic who were older and had a greater likelihood to be exposed to other interventions.  This study would tend to be classified as more of an effectiveness study.  The fact that the Australian study was a null study does not mean that the intervention was not effective in the ‘real world’.  Rather, for the patients enrolled, the treatment was not effective when used in that particular setting and context.  Perhaps you may encounter infants with colic who have feeding history and medical history more like the infants from the Italian study. Understanding the context of the studies helps identify those characteristics that may or may not apply to the infants with colic who you may treat in your clinic.


Which is better: Efficacy or Effectiveness?

When developing a new or experimental intervention, an efficacy study might be important to increase the likelihood of detecting a positive change.  However, “real world” factors may make a difference in how a product is used.  Perhaps an intervention might be inconvenient (due to multiple doses throughout the day) or unpalatable for the patient.  Perhaps the dosing regimen is complicated and the primary care providers don’t apply the correct dosing for patients. In these cases, an effectiveness study might be a better guide to how useful the intervention will be in clinical practice.

As a final note, it can be tempting to simply read the abstract of a clinical trial to assess the results of a study.  However, in many instances the crucial details of the study (e.g., how the study participants were selected, who was included or excluded, what type of clinical setting was used) are buried in the methods section of the study.  Patient diet, exposure to other treatments and comorbid conditions are all common confounding factors encountered in trials evaluating supplements.  When reading through the literature and understanding if a study is applicable to your practice, be sure to understand the full context and purpose of the study.  “Was this study useful for determining clinical efficacy or clinical effectiveness?” is an important question for readers of probiotic and prebiotic clinical trials. Keeping this question in mind may help you better resolve what may appear to be inconsistency among clinical trials.