Posts

60 Minutes’ 13 minutes on probiotics

By Mary Ellen Sanders, PhD, ISAPP Executive Science Officer 

On June 28, 60 Minutes aired a 13-minute segment about probiotics titled, “Do Probiotics Actually Do Anything?” Unfortunately the media segment did not provide listeners with a nuanced perspective.

‘Probiotics’ were treated as if they were one entity, ignoring the best approach to addressing the topic of what probiotics do: evaluate the evidence for specific strains, doses and endpoints, and then make a conclusion based on the totality of the evidence. They would have found that many experts agree that actionable evidence exists for certain probiotics to prevent antibiotic associated diarrhea (here, here), prevent upper respiratory tract infections (here), prevent morbidity and mortality associated with necrotizing enterocolitis (here,), treat colic (here), and treat acute pediatric gastroenteritis (here). (For an overall view of evidence, see here.)

Importantly, not all retail probiotics have evidence (at least evidence that is readily retrievable, see here and here). But that does not mean that none do.

The 60 Minutes segment also highlighted questions about probiotic safety. No intervention is without risk, and no one claims as much for probiotics. Prof. Dan Merenstein, MD, just one clinical investigator of probiotics, has collected over 20,000 pediatric clinical patient days’ worth of safety data over the past eight years of clinical investigation, with no indication of safety concerns. In fact, participants in the placebo group generally have more adverse events than in the probiotic groups. But importantly, the safety standard for probiotics was mischaracterized by 60 Minutes. According to Dr. James Heimbach, a food safety expert (not interviewed in the segment) who has conducted 41 GRAS determinations on probiotics, over 25 of them notified to the FDA, he objects to the statement that GRAS is a lower safety bar than a drug. He clarifies:

“The safety standard that applies to food additives and GRAS substances, “reasonable certainty of no harm,” is a far higher standard than that applying to drugs. Drugs are judged against a risk/benefit standard, which can potentially allow quite dangerous drugs on the market provided they offer a significant benefit. The safety standard for drugs also applies only to prescribed doses for specific individuals over prescribed durations. The food-additive/GRAS substance standard, on the other hand, requires safety at any biologically plausible level of intake, for any person (child, adult, elderly; pregnant; etc.), over a lifetime. And it is a risk-only standard—no potential benefit is allowed to override the “reasonable certainty of no harm” standard. Additionally, in the case of GRAS substances (which includes most probiotics), the evidence of safety must be published in the peer-reviewed scientific literature and be widely accepted by the scientific community as well as by government regulators.”

Finally, the story implied that benefits people claim for themselves when using probiotics are due to a placebo effect. This ignores the many properly controlled studies directly comparing the effects of specific probiotics to placebos. A positive trial on probiotics, such as observed in this recent trial on irritable bowel syndrome symptoms (here) and in most trials included in Cochrane meta-analyses on prevention of C. difficile-associated diarrhea (here), means that positive effects were observed beyond any placebo effect. The placebo effect is real, equally applicable to probiotics and drugs, but as with all clinically evaluated substances, properly controlled trials control for this effect.

The probiotic field has come a long way over the past 20 years with regard to number and quality of clinical trials. In that time, well-done systematic reviews of the evidence have found benefits for specific probiotics for specific conditions, while also finding a lack of evidence for beneficial effects in other contexts. There are of course well-conducted clinical trials that have failed to demonstrate benefit (here, here, here). This should not be equated to mean that probiotics do not do anything.

Many challenges remain for improving the quality of the evidence across the wide range of different strains, doses, endpoints and populations. More clinical research needs to be conducted in a manner that minimizes bias and is reported according to established standards. Confidence in the quality of commercial products could be improved by industry adopting third party verification (here), and the quality of products targeting compromised populations need to be fit for purpose (here). Companies should stop using the term ‘probiotic’ on products that have no evidence warranting that description. We need to understand much better how a person’s individual situation, such as diet, microbiome, use of medications and fitness, impact the ability of a probiotic to promote health. Much remains to be learned in this evolving and exciting field. As Dr. Merenstein says, “The key question is not, ‘Do probiotics actually do anything?’, as that is easily answered ‘yes’ when you look at robust placebo-controlled trials of specific probiotics. Better questions are ‘Which probiotics do anything, and for what?’”

Further reading:

Misleading press about probiotics: ISAPP responses

ISAPP take-home points from American Gastroenterological Association guidelines on probiotic use for gastrointestinal disorders

New publication gives a rundown on probiotics for primary care physicians

Safety and efficacy of probiotics: Perspectives on JAMA viewpoint

New publication gives a rundown on probiotics for primary care physicians

With an increasing number of patients becoming aware of the human microbiome and its role in health, primary care physicians are faced with questions about probiotics as a possible strategy for maintaining health. Patients may see conflicting messages in the news and on product labels – so how can they know which probiotic benefits are scientifically proven?

A new publication in the Journal of Family Practice provides a quick update on evidence for the use of probiotics in different indications, so primary care physicians can equip themselves to provide evidence-based recommendations and to answer patients’ most commonly asked questions about probiotics.

Written by ISAPP board members Daniel J. Merenstein, MD and Mary Ellen Sanders, PhD, along with Daniel J. Tancredi, PhD, the article provides practical advice in the form of practice recommendations, along with comments about safety data from numerous clinical trials.

As Dr. Merenstein stated, “We wrote this article for working clinicians. They are interested in the science but are busy and want a straightforward evidence-based resource. We are hopeful this will be a go-to resource during the busy clinic day.”

Verbatim from the article are the following practice recommendations:

  • Consider specific probiotics to prevent antibiotic-associated diarrhea, reduce crying time in colicky infants, and improve therapeutic effectiveness of antibiotics for bacterial vaginosis.
  • Consider specific probiotics to reduce the risk for Clostridioides (formerly Clostridium) difficile  infections, to treat acute  pediatric diarrhea, and to manage symptoms of constipation.
  • Check a product’s label to ensure that it includes the probiotic’s genus, species, and strains; the dose delivered in colony-forming units through the end of shelf life; and expected benefits.

The full text can be accessed by logging into Medscape.

ISAPP provides guidance on use of probiotics and prebiotics in time of COVID-19

By ISAPP board of directors

Summary: No probiotics or prebiotics have been shown to prevent or treat COVID-19 or inhibit the growth of SARSCoV-2. We recommend placebo-controlled trials be conducted, which have been undertaken by some research groups. If being used in clinical practice in advance of such evidence, we recommend a registry be organized to collect data on interventions and outcomes.  

Many people active in the probiotic and prebiotic fields have been approached regarding their recommendations for using these interventions in an attempt to prevent or treat COVID-19. Here, the ISAPP board of directors provides some basic facts on this topic.

What is known. Some human trials have shown that specific probiotics can reduce the incidence and duration of common upper respiratory tract infections, especially in children (Hao et al. 2015; Luoto et al. 2014), but also with some evidence for adults (King et al. 2014) and nursing home residents (Van Puyenbroeck et al. 2012; Wang et al. 2018). However, not all evidence is of high quality and more trials are needed to confirm these findings, as well as determine the optimal strain(s), dosing regimens, time and duration of intervention. Further, we do not know how relevant these studies are for COVID-19, as the outcomes are for probiotic impact on upper respiratory tract infections, whereas COVID-19 is also a lower respiratory tract infection and inflammatory disease.

There is less information on the use of prebiotics for addressing respiratory issues than there is for probiotics, as they are used mainly to improve gut health. However, there is evidence supporting the use of galactans and fructans in infant formulae to reduce upper respiratory infections (Shahramian et al. 2018; Arslanoglu et al. 2008). A meta-analysis of synbiotics also showed promise in repressing respiratory infections (Chan et al. 2020).

Mechanistic underpinnings. Is there scientific evidence to suggest that probiotics or prebiotics could impact SARS-CoV-2? Data are very limited. Some laboratory studies have suggested that certain probiotics have anti-viral effects including against other forms of coronavirus (Chai et al. 2013). Other studies indicate the potential to interfere with the main host receptor of the SARS-CoV-2 virus, the angiotensin converting enzyme 2 (ACE2). For example, during milk fermentation, some lactobacilli have been shown to release peptides with high affinity for ACE (Li et al. 2019). Recently, Paenibacillus bacteria were shown to naturally produce carboxypeptidases homologous to ACE2 in structure and function (Minato et al. 2020). In mice, intranasal inoculation of Limosilactobacillus reuteri (formerly Lactobacillus reuteri) F275 (ATCC 23272) has been shown to have protective effects against lethal infection from a pneumonia virus of mice (PVM) (Garcia-Crespo et al. 2013). These data point towards immunomodulatory effects involving rapid, transient neutrophil recruitment in association with proinflammatory mediators but not Th1 cytokines. A recent study demonstrated that TLR4 signaling was crucial for the effects of preventive intranasal treatment with probiotic Lacticaseibacillus rhamnosus (formerly Lactobacillus rhamnosus) GG in a neonatal mouse model of influenza infection (Kumova et al., 2019). Whether these or other immunomodulatory effects, following local or oral administration, could be relevant to SARS-CoV-2 infections in humans is at present not known.

Our immune systems have evolved to respond to continual exposure to live microbes. Belkaid and Hand (2016) state: “The microbiota plays a fundamental role on the induction, training, and function of the host immune system. In return, the immune system has largely evolved as a means to maintain the symbiotic relationship of the host with these highly diverse and evolving microbes.” This suggests a mechanism whereby exposure to dietary microbes, including probiotics, could positively impact immune function (Sugimura et al. 2015; Jespersen et al. 2015).

The role of the gut in COVID-19. Many COVID-19 patients present with gastrointestinal symptoms and also suffer from sepsis that may originate in the gut. This could be an important element in the development and outcome of the disease. Though results from studies vary, it is evident that gastrointestinal symptoms, loss of taste, and diarrhea, in particular, can be features of the infection and may occur in the absence of overt respiratory symptoms. There is a suggestion that gastrointestinal symptoms are associated with a more severe disease course. Angiotensin converting enzyme 2 and virus nucleocapsid protein have been detected in gastrointestinal epithelial cells, and infectious virus particles have been isolated from feces. In some patients, viral RNA may be detectable in feces when nasopharyngeal samples are negative. The significance of these findings in terms of disease transmission is unknown but, in theory, do provide an opportunity for microbiome-modulating interventions that may have anti-viral effects (Cheung et al. 2020; Tian et al. 2020; Han et al. 2020).

A preprint (not peer reviewed) has recently been released, titled ‘Gut microbiota may underlie the predisposition of healthy individuals to COVID-19’ (Gao et al. 2020) suggesting that this could be an interesting research direction and worthy of further discussion. A review of China National Health Commission and National Administration of Traditional Chinese Medicine guidelines also suggested probiotic use, although more work on specific strains is needed (Mak et al. 2020).

Are probiotics or prebiotics safe? Currently marketed probiotics and prebiotics are available primarily as foods and food/dietary supplements, not as drugs to treat or prevent disease. Assuming they are manufactured in a manner consistent with applicable regulations, they should be safe for the generally healthy population and can be consumed during this time.

Baud et al. (in press) presented a case for probiotics and prebiotics to be part of the management of COVID-19. Although not fully aligned with ISAPP’s official position, readers may find the points made and references cited of interest.

Conclusion. We reiterate, currently no probiotics or prebiotics have been shown to prevent or treat COVID-19 or inhibit the growth of SARSCoV-2.

 

Safety and efficacy of probiotics: Perspectives on JAMA viewpoint

By Mary Ellen Sanders PhD, executive science officer, ISAPP,  and Daniel Merenstein MD, Department of Family Medicine, Georgetown University School of Medicine

The Journal of the American Medical Association (JAMA) recently published a short viewpoint that called into question the safety and efficacy of probiotics. After careful review, we concluded that some opinions expressed were not consistent with available data. We share our perspectives here.

Claim 1: The paucity of high-quality data supporting the value of probiotics.

The authors speak to the “paucity” and “lack” of data supporting probiotic use. They criticize probiotic meta-analyses in general, even though there are many well-done ones, which describe clear PICOS, assess the quality of studies included, and assess publication bias. Many conclude that there is evidence that certain probiotics may be beneficial for several clinical endpoints. In the case of treatment of colic, an individual participant data meta-analysis was conducted on a single strain, and concluded “L reuteri DSM17938 is effective and can be recommended for breastfed infants with colic” (Sung et al. 2018). For necrotizing enterocolitis (NEC), a change in practice is recommended by a Cochrane meta-analysis (AlFaleh et al. 2018), which is consistent with draft American Gastroenterological Association (AGA) recommendations posted last month. In some cases, conclusions are qualified as being based on low quality data, which is also the case with many standard-of-care medical interventions. Other benefits supported for certain probiotics by evidence are shown in Table 1 of Sanders et al. 2018. But an evidence-based review of available data would not support a general statement that “data are lacking.”

Instead, we think a discussion of what evidence is actionable is reasonable to have. For this discussion, different people or groups can reasonably set the bar at different levels for what constitutes actionable evidence. But several medical organizations, including the European Society for Paediatric Gastroenterology, Hepatology and Nutrition, World Gastroenterology Organisation, American College of Gastroenterology, AGA (proposed, for antibiotic-associated diarrhea, NEC and pouchitis), European Crohn’s and Colitis Organization, and European Society for Primary Care Gastroenterology have actionable recommendations for probiotic use for one or more indications. For those indications, any individual physician may judge that the available evidence as not convincing to him or her, but many qualified healthcare experts did find the evidence convincing and have made recommendations accordingly. We recognize that the JAMA viewpoint was limited in the number of words and references allowed, but to impugn an entire field, the authors are obliged to explain why their views differ so much from established organizations.

The authors also criticize the inclusion of small, single-center trials in probiotic meta-analyses. They state such studies have less oversight, are more susceptible to misconduct and are at greater risk of bias than larger, multicenter trials, and thereby skew conclusions of meta-analyses in favor of probiotics. They state, without evidence, that small trials are more likely to show large effects and are more likely to be published. They advocate for meta-analyses that only include multi-center trials, thereby ignoring much available evidence on the basis of unsubstantiated preferences. There are a number of reasons why some trials are multi-center, but improved quality or closer monitoring are not among them (see here, here and here). Multicenter trials may be necessary to study a rare medical endpoint, a condition with an expected small effect size but significant health implications, or to accelerate the time course for a study. In fact, an analysis of 81 meta-analyses of RCTs in 2012 concluded:

“Our results do not support prior findings of larger effects in SC (single-center) than MC (multi-center) trials addressing binary outcomes but show a very similar small increase in effect in SC than MC trials addressing continuous outcomes. Authors of systematic reviews would be wise to include all trials irrespective of SC vs. MC design and address SC vs. MC status as a possible explanation of heterogeneity (and consider sensitivity analyses).” [Emphasis ours]

 

In our experience, the size of a study does not inevitably minimize risk of bias. We have directly witnessed private physicians enroll for large multi-site trials without such oversight or professionalism. As the great David Sackett said in his paper from 20 years ago, “The more detailed the entry form and eligibility criteria for ‘somebody else’s’ RCT, the greater the risk the criteria will be ignored, misunderstood or misapplied by distracted clinicians who regard them as further intrusions into an overfull call schedule.” Further, due to often being underpowered, taken alone smaller studies are less, not more, likely to generate positive findings than larger trials. But when they are included in a meta-analysis, these studies contribute to the total body of evidence. We have personally worked on many single-center randomized controlled trials on probiotics. These often have monitors from the U.S. Food and Drug Administration and/or the National Institutes of Health, they are all registered with both primary and secondary outcomes listed, they utilize a data safety monitoring board, they undergo true allocation concealment, and otherwise are conducted to minimize risk of bias. To criticize probiotic studies for being single-center vs multicenter seems unjustified and baseless.

It is quite true that many of the studies conducted on probiotics were done 15 or more years ago, and the quality standards do not meet what we expect today. We wholeheartedly agree but would ask the authors to review studies conducted on drugs 15 years ago, and they will see the same issues. So we agree that more trials using modern quality standards are needed in the field of probiotics, as is the case for any interventions with a long history of being studied.

Claim 2: Potentially biased reviews of probiotic efficacy

In trying to explain why physicians might recommend probiotics, the authors speculate that some professional societies and some journals may be insufficiently critical in reviewing probiotic studies due to financial conflicts of interest. We have no doubt that there is bias in the scientific realm, which is not just limited to financial conflicts of interest, but question if there is any evidence that this occurs any more or less frequently with probiotics compared to any other realm of science. To leverage this accusation at the probiotic field specifically implies it is especially egregious, but no data supporting this accusation were provided. Also there is no face validity for this accusation. There is much more money to be made by pharmaceuticals and medical interventions than probiotic supplements and yogurts.

Claim 3: Complex framework in which probiotics are regulated and sold

The regulatory framework for probiotics can be difficult to navigate and is not always in the best interest of stakeholders, but we don’t think it’s reasonable to criticize the probiotic field for this situation. In the USA, probiotic products are bound by law that was enacted by Congress and the rules/guidance developed by the FDA for allowable product claims, levels of required regulatory oversight, and lack of requirements for premarket approval. It is fair to criticize Congress and the FDA for these circumstances surrounding the category of dietary supplements, but doing this in the context of an article on probiotics unfairly maligns probiotics.

Drugs vs dietary supplements. Most probiotics are sold as foods or dietary supplements. Since probiotics were first described as fermenting microbes in soured milk, this makes historical sense. Companies and consumers do not view these products as drugs, and in most cases they are not used as drugs. Outside a regulatory mindset, it makes perfect sense for foods to be useful for promoting health and managing symptoms, and this is what has driven 30 years of research and marketing of probiotics. Forcing all probiotics into a drug rubric would deprive consumers of access, would greatly increase their cost, and would preclude responsible food/supplement manufacturers from producing them. Drugs are drugs primarily to protect the safety of the patient. All drugs are assessed with a risk/benefit balance, and in some cases, the risk is significant. In the case of probiotics, we agree with the authors that most probiotics are likely safe for the general population. We see no reasonable justification to advocate that these products must all be researched and sold as drugs.

Probiotic product quality.The authors seem to prefer the drug model for probiotics based on a perceived need for improved product quality and oversight. Yet all foods and dietary supplements in the USA are required by law to be manufactured under good manufacturing practices. This includes most every product bought at the grocery store and served for dinner as well as probiotic foods and supplements. Further, companies are required to label their products in a truthful and not misleading fashion, including representations of contents and claims. Companies that fail to meet these standards are in violation of the law. Yes, there are products – of all types – that fall short of these requirements. The many responsible probiotic manufacturers and probiotic scientists decry such occurrences. However, these cases do not define the probiotic field any more than medical errors define physicians. It is not fair to impugn the entire probiotic industry based on the ‘bad apples’ that participate in it. A 2017 ESPGHAN review cites surveys of probiotic products from different regions globally, most of which report examples of probiotic products falling short in some quality attribute. Such surveys highlight quality problems, but due to sampling and methodological approaches, their results do not provide a reliable estimate of the extent of problem among commercial probiotic products. Many probiotic products are produced responsibly and are subjected to third party quality audits. The absence of such third party documentation is not evidence of poor quality, but we agree that it serves to improve consumer and healthcare provider confidence (see Jackson et al. 2019), and if more fully adopted, would weed out irresponsible probiotic manufacturers.

Oversight of probiotic research. The authors state, “If a manufacturer claims that any product, including a probiotic, cures, mitigates, treats, or prevents disease, the product is classified as a drug, thereby triggering a costly Investigational New Drug (IND) application process.” However, they seem to conflate the regulatory approach to product claims and the regulatory oversight of biologic drug research. In the case of product claims, if a product claims to cure, treat, prevent or mitigate disease, it is by definition a drug. If it has not undergone appropriate drug approval process, it is an illegally marketed drug and is subject to FDA action, including recall. Probiotics not destined for sale as drugs should not have to be researched under a drug rubric. This does NOT mean that such studies will de facto be substandard studies. We all understand the importance of conducting and reporting trials according to well-established guidelines. Studies on foods and supplements can and should follow those same principles.

Claim 4: Possible concerns about probiotic safety

Medical professionals balance potential harm with potential benefit for any intervention they recommend. Regarding safety of probiotics, the authors acknowledge that most probiotics are likely safe, but we would qualify that statement with “for their intended uses.” The use of probiotics in critically ill patient populations needs to be done with caution, proper oversight and a justification that the potential benefit will outweigh risk. The authors cite two examples to support their concern about probiotic safety, both in critically ill patient populations. One was a retrospective study looking at bacteremia in critically ill children (see the report here and responses to the report here and here). The second was a RCT that reported higher mortality in patients with pancreatitis (see the report here, with additional perspectives on interpreting safety outcomes here and here). We are not aware of any probiotics that are marketed for such uses, and if they were, they would be marketed as drugs, requiring drug-level safety and efficacy evidence. These studies are not an indictment of safety of probiotic foods and supplements, which in most cases are intended for the generally healthy population.

The authors further state that studies identifying adverse events from probiotics are the “tip of the iceberg” – creating an image of a huge number of unreported adverse incidents poised to be revealed. We have personally studied the most widely used Bifidobacterium strain, and in well over 30,000 pediatric patient days have not seen any serious adverse events and no more adverse events than placebo. The article cited by the authors states that our trials adequately reported harm. Obviously, no intervention is harmless, and no one claims as much for probiotics. It is true that older probiotic studies can rightly be criticized for not rigorously collecting and reporting data on adverse events (Hempel et al. 2011). However, a reasonable assessment of all available data, including data from well-conducted clinical trials, including trials in vulnerable populations, history of safe use, FDA notified assessments for GRAS use of certain probiotic strains, European Food Safety Authority QPS list, and others support that commonly used probiotics have a strong safety record for use in the general public.

Transferable antibiotic resistance. Regarding the risk that probiotics may transfer antibiotic resistance genes, this is a hypothetical concern – there is no documented case of this. Further, one pillar of probiotic safety assessments is that strains with antibiotic resistance genes flanked by mobile genetic elements are excluded from commercialization. As stated by Ouwehand et al. 2016, “Probiotics are specifically selected to not contribute to the spread of antibiotic resistance and not carry transferable antibiotic resistance.” The current approach to probiotic safety is that complete, well annotated genome sequences are available for commercial strains. This information is typically included in GRAS notices submitted to the FDA, and all the major probiotic suppliers require this level of safety assessment. This is the expected standard by the European Food Safety Authority as well, a standard that we enthusiastically and unreservedly endorse. Transferable antibiotic resistance is not a lurking threat of probiotics use, but is a well-considered issue adequately addressed by responsible probiotic manufacturers.

Conclusion

We believe that this JAMA viewpoint misrepresents the totality of data on probiotics and can potentially do harm by dissuading use of probiotics in an evidence-based manner. Important points have been raised by the authors, especially with regard to the use of probiotics in vulnerable populations, but this does not characterize most of probiotic use. We agree, as we expect the majority of scientists working on probiotics would, that additional, well controlled human studies are needed. That was why we were pleased to see the authors’ studies assessing the impact of L. rhamnosus R0011 and L. helveticus R0052 or L. rhamnosus GG on acute pediatric gastroenteritis, even though the results of both studies were null (see blog post regarding these studies here and here). But as we await additional trials, we have a responsibility to consider available evidence. The authors raise many good points that the entire medical field could learn from, but there are clear indications for probiotics and they should continue to be used for these indications, likely benefitting many while harming few.

Acknowledgements

MES and DM are grateful for the critical review of this perspective by probiotic safety expert Dr. James Heimbach, biostatistician Dr. Daniel Tancredi, and gastroenterologist and probiotic expert Dr. Eamonn Quigley.

 

 

 

Probiotics in fridge

The FDA’s view on the term probiotics, part 2: Further down the rabbit hole

By James Heimbach, Ph.D., F.A.C.N., JHEIMBACH LLC, Port Royal, VA

A number of weeks ago I wrote on the ISAPP blog about US Food and Drug Administration (FDA) declining to file Generally Regarded As Safe (GRAS) notices that described the subject microorganism as a “probiotic” or “probiotic bacterium” (see The FDA’s view on the term “probiotics”). Now the FDA’s response to such GRAS notices has developed additional ramifications. Let me put them into two categories: Class 1 misdemeanors that will cause FDA to reject the notice, and Class 2 misdemeanors that will probably not prevent filing, but will cause FDA to raise questions. I should note that these thoughts are based on both my own direct experiences and my repeated telephone conference calls with FDA.

Class 1 Misdemeanors

  1. Using the term probiotic in any way in describing or characterizing the subject microorganism or its past, present, or intended use.
  2. Extended discussion of benefits derived from ingestion of the microorganism in animal or human research.
  3. Any mention, however brief, of the potential for the microorganism to be used in dietary supplements.

Class 2 Misdemeanors

  1. Including brief mentions of the microorganism serving as a probiotic. E.g., if you cite a study of the microorganism that you might previously have reported as “a study of the probiotic benefits” of the microorganism, change it to simply “a study of the benefits” of the microorganism. This same caution is advised when reporting opinions from the European Food Safety Authority (EFSA) or other authoritative bodies.
  2. Using the word “dose” in describing intended use. Also see #4 below.
  3. Virtually any use of the term “dietary supplement,” including in reporting past, current, or intended uses of the strain or the species in Europe or elsewhere, by anyone.
  4. Even relatively brief mentions of benefits. The recommended way of handling reporting of human studies of the species or strain is to avoid any narrative at all. Simply summarize the studies in tabular form, listing the citation, study design (RCT, open-label, etc.) and objective, study population (number, sex, age, characterization such as IBS patients, malnourished children, preterm infants), test article (microorganism binomial and strain), dose (but call it “administration level”—“dose” can be seen as indicating a drug or dietary supplement), duration, and safety-related results. Include methods used to ensure that any adverse events or severe adverse events would have been reported—medical examinations, self-report questionnaires, parental questionnaires, biochemical measures, etc.—and at what time points during or after the in-life portion of the research. Avoid ANY discussion of improvements seen in the test group.

Good luck!

A Miracle Treatment! Or Not?

By Daniel J. Merenstein, MD, Professor, Department of Family Medicine and Director of Research Programs, Georgetown University Medical Center, Washington DC

Here’s a scenario for a physician: A drug rep walks into your office. She has a new product she wants to talk to you about. You are super excited to talk to her as you have heard all about this product from many other sources. The data that are being reported are amazing. There are hundreds if not many more case reports of it working. People were dying and then totally recovered after being given this product. It has been witnessed and published! The efficacy is well over 90%. You are not sure there is any intervention you have ever heard of that has such amazing efficacy.  She tells you that in some of the cases, the patients were very sick and despite numerous courses of antibiotics they did not improve until this new product was given. You ask for more information as you are starting to think this must be like when doctors first heard of penicillin.

The product can be taken orally but that is not the way it is generally given. She tells you that although there are 2-3 ways to administer, most hospitals are doing it the most expensive way now. (You later learn that the typical– and most expensive – approach to administering the product may not even be the best approach.) But you withhold judgement as this sounds exciting. And remember, you have been hearing all about this from so many different sources.

But as you listen, it gets a little confusing. She tells you that the makeup of the product is different in nearly every application. This makes it exciting to use, as one really never knows what is in it. It is also relatively cheap to obtain, as the patient can have a friend just bring it in for them.

Since you are trained in evidence-based medicine, you ask a few questions. It is exciting there are all sorts of case reports but what about the randomized controlled trials, and what does the FDA say about it? You ask if you can look at the trials—there is no way you can review hundreds of studies now but if she leaves them for you, you will look at them this weekend. But before she leaves you ask a few quick questions. How many of these studies are randomized? She says 10. How many use a placebo? She says 6. You tell her what you really want to do is review all the randomized placebo-controlled blinded studies, if she can just leave those.

Later in the week you go pick up the folder she left and right away are a little surprised at how light it is. It looks like there are only 3 randomized placebo-controlled blinded studies, only two of which are peer-reviewed and published. One was a positive study; overall, 91% of patients in the new drug group achieved clinical cure compared with 63% in the control group. But you realize this is not exactly a placebo-controlled trial. What they did is compare two types of the new application. Furthermore, this study was conducted at two sites and at one of the sites both the new application and the control had nearly identical rates of improvement, both over 90%. Okay so this was not a perfect study, only 46 total participants, but still pretty exciting with over 90% improvement.

The second study had three groups of 83 people. Group A (2 doses of new drug), B (2 doses of placebo) and C (1 dose of new drug and 1 placebo dose). The efficacy for these three groups was 61%, 45%, and 67%, respectively. The primary endpoint was not met (P = .152). Interestingly, Group C, which included one dose of placebo, was superior to all placebo (group B) but Group A, in which the drug was given two times, was not superior to placebo.

The third study, a Phase II trial, appears to not be peer-reviewed or published, but just reported online. However, it does appear this was far from a positive study, with 44% of subjects (26 of 59) who received the new application improving versus 53% of subjects (16 of 30) who received placebo. I have been told that this study will be published soon and that a Phase III study of this intervention was also undertaken.

Well now you are getting a little more confused. You have heard from fellow docs, the lay press, medical literature and the drug rep that this new application was over 90% effective. But it appears in the three reasonably well controlled studies, the ones from which we can really draw conclusions, only one was positive and in that study the control was not a real placebo.

Besides efficacy, you remember that one has to always consider the cost and adverse events. Maybe this new application is like recommending the Mediterranean Diet, where the efficacy from studies is limited but the adverse events are nearly non-existent. But when you do a quick PubMed search you learn that this is far from the case with this product. This application has been reported to cause very serious adverse events, including extended-spectrum beta-lactamase (ESBL)–producing Escherichia coli bacteremia resulting in one death. You look online expecting that the FDA must have some serious warnings about this new drug. You don’t find any such warnings.

You may have guessed that the product is in fact a Fecal Microbiota Transplant (FMT). Besides having a professional interest in this much-discussed treatment, I have a personal interest. Last year my son was in a Johns Hopkins Hospital with a central line and two broad-spectrum antibiotics for a bone infection. I asked them to provide him with probiotics since the number needed to treat to prevent pediatric antibiotic associated diarrhea is 9, per a 2019 Cochrane review. This review included 20 randomized, placebo-controlled studies of a single strain. However, I was told no Hopkins hospital will administer probiotics, and further, that we could not even bring in our own because of concerns for the safety of others. But no worries – if my son got recurrent C. diff infection, Hopkins would allow this great new procedure, FMT.

In medicine I cannot truly imagine a probiotic with the same evidence base as FMT receiving such widespread acceptance and escaping regulatory scrutiny. And currently used probiotics have an excellent safety record. Just imagine, if this were a new drug being sold there would be widespread condemnation of the attempt to get approval mainly based on anecdotal case reports.  Shockingly, based on the level of evidence I have described many experts now think a randomized placebo-controlled trial is not even ethical for the placebo group, as of course they know FMT works.

It is a quandary. I am not opposed to FMT; I find it fascinating. But why has it been so widely accepted and why has the FDA, which in general has been very careful with probiotic applications in medicine, allowed this to proceed for recurrent C diff infection with only enforcement discretion? Both treatments administer live microorganisms, one with 31 placebo controlled randomized trials, including 8672 subjects [of C. diff prevention (number needed to prevent=42), not treatment like FMT], the other with pretty limited data.  I have my thoughts, but better for you to ponder it.

 

 

Misleading press about probiotics: ISAPP responses

By Mary Ellen Sanders, PhD, Executive Science Officer, ISAPP

It seems over the last couple of years, open season on “probiotics” has been declared. Responding in a scientifically accurate fashion to misleading coverage, whether it is in reputable scientific journals or in the lay media, takes time and care.

I want to be clear: well-conducted clinical trials, regardless of the outcomes, are welcome contributions to the body of evidence. No one expects that every probiotic will work for every indication. Null trials document this – they tell researchers to look elsewhere for solutions. Further, we must acknowledge the limitations and weaknesses of available evidence; unfortunately, not all trials are well-conducted. We also need to be just as diligent in criticizing press that is overly positive about probiotic benefits, which are not backed by evidence.

However, articles with misleading information are all-too-frequently published. Below are ISAPP responses to some of these stories.

  1. A paper on rhamnosus GG bacteremia in ICU patients led to headlines about ‘deadly infections’ and probiotic administration ‘backfiring’, even though no patients died and clinical outcomes were not collected. ISAPP responded to clarify appropriate context for understanding the safety issues raised from this paper. See Lactobacillus bacteremia in critically ill patients does not raise questions about safety for general consumers.
  2. The Wall Street Journal published an article condemning probiotics for reducing fecal microbial diversity. ISAPP responded with a blog Those probiotics may actually be helping, not hurting, pointing out the errors in the author’s thinking (equating diversity with gut health).
  3. A pair of well-conducted clinical trials that did not show impact of probiotics on pediatric acute diarrhea led to some ignoring all previous evidence and concluding that no probiotics were useful for acute pediatric diarrhea. ISAPP responded about the importance of putting new evidence in the context of the totality of evidence: L. rhamnosus GG for treatment of acute pediatric diarrhea: the totality of current evidence. Also, Dr. Eamonn Quigley, an ISAPP board member, published an independent response.
  4. Pieter Cohen concluded that evidence for probiotic safety is insufficient in an article in JAMA Internal Medicine. ISAPP’s response was published in a letter to the editor, along with Cohen’s response to our letter.
  5. Responding to two papers in Cell (here and here), and accompanying media coverage that called into question probiotic safety and efficacy, ISAPP published a detailed post Clinical evidence and not microbiota outcomes drive value of probiotics objecting to conclusions, and released a public statement.
  6. Jennifer Abbasi wrote a critical article about probiotics with the inflammatory title “Are Probiotics Money Down the Toilet? Or Worse?” ISAPP responded with the following blog post: Probiotics: Money Well-Spent For Some Indications.
  7. When Rao, et al incriminated probiotics as a cause of D-lactic acidosis, ISAPP posted a blog and published a letter to the editor of Clin Transl Gastroenterol objecting to this conclusion.
  8. ISAPP responded to a paper claiming that probiotics were unsafe in children: Probiotics and D-lactic acid acidosis in children and Brain Fogginess and D-Lactic Acidosis: Probiotics Are Not the Cause.

Board member and Professor Colin Hill wrote a blog post called Another day, another negative headline about probiotics? His post provides some useful questions to consider when reacting to a publication:

  • Is the article describing an original piece of research and was it published in a reputable, peer-reviewed journal?
  • What evidence is there that the strain or strain mix in question is actually a probiotic? Does it fit the very clear probiotic definition?
  • Was the study a registered human trial? How many subjects were involved? Was it blinded and conducted to a high standard?
  • What evidence was presented of the dose administered and was the strain still viable at the time of administration?
  • Were the end points of the study clear and measurable? Are they biologically or clinically significant to the subjects?
  • Did the authors actually use the words contained in the headline? “Useless”, or “waste of money”, etc?

 

Lactobacillus bacteremia in critically ill patients does not raise questions about safety for general consumers

By Dan Merenstein MD, Professor of Family Medicine, Georgetown University Medical Center, Washington DC, USA; Eamonn Quigley MD, Professor of Medicine, Houston Methodist Hospital and Weill Cornell Medical College, Texas USA; Gregory Gloor PhD, Professor of Biochemistry, Schulich School of Medicine and Dentistry, The University of Western Ontario, Canada; Hania Szajewska MD, Professor of Paediatrics, The Medical University of Warsaw, Poland;  and Mary Ellen Sanders PhD, Executive Science Officer, ISAPP, Colorado, USA.

A recent Nature Medicine paper reported blood cultures positive for L. rhamnosus GG in six critically ill patients at Children’s Hospital in Boston.

About this study

Patients (aged 1, 2, 4, 12, 19 and 19 years) with L. rhamnosus-associated bacteremia suffered from different chronic conditions (mitochondrial disorder, cerebral palsy, congenital heart disease, cystic fibrosis) and were located either in the ICU (cardiac or medical/surgical ICU) or ICP (intermediate care program) at the hospital at the time of bacteremia. The bacteremia was discovered during routine blood culture screens. Clinical presentations were not described in detail; however, none had endocarditis or died from the bacteremia, although one did get a central line infection. Two of the six cases of bacteremia in probiotic-consuming patients were determined by attending physicians to be transient or due to contaminants, and were not treated. The other four cases were treated with antibiotics. A further 516 patients dosed with the same probiotic did not develop bacteremia.

The researchers examined the blood isolates and using whole genome sequencing were able to confirm that the Lactobacillus isolated from the blood of these patients was genetically identical – with the exception of a few SNPs – to L. rhamnosus GG present in the probiotic product. This is the preferred approach to confirming the source of blood culture isolates.

Important questions arising from critical review of this paper

  1. Was the study appropriately controlled?

The authors report a seemingly high rate (1.1%) of Lactobacillus bacteremia among the 522 L. rhamnosus-consuming patients compared with 0.009%, the rate of Lactobacillus bacteremia among 21,652 patients who did not receive probiotics.  However, the paper does not justify the legitimacy of comparing these two groups to each other. Indeed, other underlying factors could contribute to the different rates of bacteremia, as these were not matched cohorts. It is important to recognize the limitations of the retrospective design used here, which limits the ability to match controls, and to control for cofounders such as underlying illness, severity of clinical illness and co-therapies (including antibiotics).

  1. What is the mechanism of transmission of the probiotic to the patients’ blood?

Most of the patients had a central line venous catheter. The paper reported that probiotics were mostly administered via tube feeding. If a probiotic is able to readily translocate the gut barrier in such patients, this would be a safety concern. But if the observed bacteremia was due to contamination of a central line, this may say more about hospital procedures than safety of the probiotic. Indeed, 16 years ago, central line contamination leading to fungemia was reported. In a 2005 paper, 92% of cases of fungemia associated with Saccharomyces cerevisiae var boulardii administration had an IV catheter. Based on such reports, handling dried probiotics in a hospital environment with critically ill patients should be done with caution. However, with proper administration procedures, certain probiotics are medically recommended in this setting.

  1. What was the clinical impact of administration of L. rhamnosus GG?

Important clinical parameters such as all-cause mortality (the outcome of greatest importance), length of hospital stay, abscesses, required medications, and others were not reported (although central line infection was reported) for the patients studied. The clinical context of this study would be more easily understood if information on the indications driving probiotic administration was provided. The authors question the risk/benefit of probiotic administration to ICU patients in a children’s hospital yet focus solely on risk and do not measure benefit. This suggests an underlying assumption by the authors that when it comes to probiotics, any risk is too much. Did the patients given L. rhamnosus GG suffer negative clinical outcomes more often than age and condition-matched controls? If so, then giving this probiotic to these patients cannot be recommended. But if not, then even though risk of bacteremia may be higher, if the patients given the probiotic fared better than matched patients, then the probiotic should be considered a reasonable option.

Lastly, the finding of a rate of Lactobacillus bacteremia of 1.1% needs to be viewed in the context of a 20% rate of nosocomial infections in the ICU (here and here).

Lessons regarding probiotic safety

Two main issues are raised by this study. The first is whether the evidence suggests opportunistic pathogenic properties of L. rhamnosus GG or rather that procedures used to administer probiotics in the ICU environment resulted in contamination, which caused bacteremia. No conclusions can be made from this study regarding this. The second is the importance of placing the results of this study into a clinical framework. The study implies risk from probiotic administration, even though the study was not powered for clinical outcomes and could not place any perceived increased risk into the context of any achieved benefit. Further, reporting rates of Lactobacillus bacteremia between cohorts unmatched for important characteristics except for probiotic use does not inform on relative risk.

Importantly for the broader situation of probiotic use, the ICU population is not reflective of the general population, so this study does not allow us to draw conclusions about safety of L. rhamnosus GG use in non-ICU patients.

We recognize the value of careful tracking of potential probiotic-associated infections and appreciate the application of bacterial genomic sequencing to identify the probiotic in the blood. Used more widely, this approach could resolve many purported claims of probiotic bacteremia.

This paper serves as an important reminder that use of probiotics in critically ill patients must be carefully considered and practice must align with learnings from the past, including the risk of central line contamination with probiotics. In addition, this paper highlights the importance of knowing the exact strain (including its antibiotic resistance profile and preferentially also its genome sequence), so that in the rare case of bacteremia, appropriate antibiotics can be administered.

See related article: Hill, C. Balancing the risks and rewards of live biotherapeuticsNat Rev Gastroenterol Hepatol (2019)

See here for an additional related open-access publication: Probiotic use in at-risk populations. Sanders et al. 2014.