Posts

A Miracle Treatment! Or Not?

By Daniel J. Merenstein, MD, Professor, Department of Family Medicine and Director of Research Programs, Georgetown University Medical Center, Washington DC

Here’s a scenario for a physician: A drug rep walks into your office. She has a new product she wants to talk to you about. You are super excited to talk to her as you have heard all about this product from many other sources. The data that are being reported are amazing. There are hundreds if not many more case reports of it working. People were dying and then totally recovered after being given this product. It has been witnessed and published! The efficacy is well over 90%. You are not sure there is any intervention you have ever heard of that has such amazing efficacy.  She tells you that in some of the cases, the patients were very sick and despite numerous courses of antibiotics they did not improve until this new product was given. You ask for more information as you are starting to think this must be like when doctors first heard of penicillin.

The product can be taken orally but that is not the way it is generally given. She tells you that although there are 2-3 ways to administer, most hospitals are doing it the most expensive way now. (You later learn that the typical– and most expensive – approach to administering the product may not even be the best approach.) But you withhold judgement as this sounds exciting. And remember, you have been hearing all about this from so many different sources.

But as you listen, it gets a little confusing. She tells you that the makeup of the product is different in nearly every application. This makes it exciting to use, as one really never knows what is in it. It is also relatively cheap to obtain, as the patient can have a friend just bring it in for them.

Since you are trained in evidence-based medicine, you ask a few questions. It is exciting there are all sorts of case reports but what about the randomized controlled trials, and what does the FDA say about it? You ask if you can look at the trials—there is no way you can review hundreds of studies now but if she leaves them for you, you will look at them this weekend. But before she leaves you ask a few quick questions. How many of these studies are randomized? She says 10. How many use a placebo? She says 6. You tell her what you really want to do is review all the randomized placebo-controlled blinded studies, if she can just leave those.

Later in the week you go pick up the folder she left and right away are a little surprised at how light it is. It looks like there are only 3 randomized placebo-controlled blinded studies, only two of which are peer-reviewed and published. One was a positive study; overall, 91% of patients in the new drug group achieved clinical cure compared with 63% in the control group. But you realize this is not exactly a placebo-controlled trial. What they did is compare two types of the new application. Furthermore, this study was conducted at two sites and at one of the sites both the new application and the control had nearly identical rates of improvement, both over 90%. Okay so this was not a perfect study, only 46 total participants, but still pretty exciting with over 90% improvement.

The second study had three groups of 83 people. Group A (2 doses of new drug), B (2 doses of placebo) and C (1 dose of new drug and 1 placebo dose). The efficacy for these three groups was 61%, 45%, and 67%, respectively. The primary endpoint was not met (P = .152). Interestingly, Group C, which included one dose of placebo, was superior to all placebo (group B) but Group A, in which the drug was given two times, was not superior to placebo.

The third study, a Phase II trial, appears to not be peer-reviewed or published, but just reported online. However, it does appear this was far from a positive study, with 44% of subjects (26 of 59) who received the new application improving versus 53% of subjects (16 of 30) who received placebo. I have been told that this study will be published soon and that a Phase III study of this intervention was also undertaken.

Well now you are getting a little more confused. You have heard from fellow docs, the lay press, medical literature and the drug rep that this new application was over 90% effective. But it appears in the three reasonably well controlled studies, the ones from which we can really draw conclusions, only one was positive and in that study the control was not a real placebo.

Besides efficacy, you remember that one has to always consider the cost and adverse events. Maybe this new application is like recommending the Mediterranean Diet, where the efficacy from studies is limited but the adverse events are nearly non-existent. But when you do a quick PubMed search you learn that this is far from the case with this product. This application has been reported to cause very serious adverse events, including extended-spectrum beta-lactamase (ESBL)–producing Escherichia coli bacteremia resulting in one death. You look online expecting that the FDA must have some serious warnings about this new drug. You don’t find any such warnings.

You may have guessed that the product is in fact a Fecal Microbiota Transplant (FMT). Besides having a professional interest in this much-discussed treatment, I have a personal interest. Last year my son was in a Johns Hopkins Hospital with a central line and two broad-spectrum antibiotics for a bone infection. I asked them to provide him with probiotics since the number needed to treat to prevent pediatric antibiotic associated diarrhea is 9, per a 2019 Cochrane review. This review included 20 randomized, placebo-controlled studies of a single strain. However, I was told no Hopkins hospital will administer probiotics, and further, that we could not even bring in our own because of concerns for the safety of others. But no worries – if my son got recurrent C. diff infection, Hopkins would allow this great new procedure, FMT.

In medicine I cannot truly imagine a probiotic with the same evidence base as FMT receiving such widespread acceptance and escaping regulatory scrutiny. And currently used probiotics have an excellent safety record. Just imagine, if this were a new drug being sold there would be widespread condemnation of the attempt to get approval mainly based on anecdotal case reports.  Shockingly, based on the level of evidence I have described many experts now think a randomized placebo-controlled trial is not even ethical for the placebo group, as of course they know FMT works.

It is a quandary. I am not opposed to FMT; I find it fascinating. But why has it been so widely accepted and why has the FDA, which in general has been very careful with probiotic applications in medicine, allowed this to proceed for recurrent C diff infection with only enforcement discretion? Both treatments administer live microorganisms, one with 31 placebo controlled randomized trials, including 8672 subjects [of C. diff prevention (number needed to prevent=42), not treatment like FMT], the other with pretty limited data.  I have my thoughts, but better for you to ponder it.

 

 

Misleading press about probiotics: ISAPP responses

By Mary Ellen Sanders, PhD, Executive Science Officer, ISAPP

It seems over the last couple of years, open season on “probiotics” has been declared. Responding in a scientifically accurate fashion to misleading coverage, whether it is in reputable scientific journals or in the lay media, takes time and care.

I want to be clear: well-conducted clinical trials, regardless of the outcomes, are welcome contributions to the body of evidence. No one expects that every probiotic will work for every indication. Null trials document this – they tell researchers to look elsewhere for solutions. Further, we must acknowledge the limitations and weaknesses of available evidence; unfortunately, not all trials are well-conducted. We also need to be just as diligent in criticizing press that is overly positive about probiotic benefits, which are not backed by evidence.

However, articles with misleading information are all-too-frequently published. Below are ISAPP responses to some of these stories.

  1. A paper on rhamnosus GG bacteremia in ICU patients led to headlines about ‘deadly infections’ and probiotic administration ‘backfiring’, even though no patients died and clinical outcomes were not collected. ISAPP responded to clarify appropriate context for understanding the safety issues raised from this paper. See Lactobacillus bacteremia in critically ill patients does not raise questions about safety for general consumers.
  2. The Wall Street Journal published an article condemning probiotics for reducing fecal microbial diversity. ISAPP responded with a blog Those probiotics may actually be helping, not hurting, pointing out the errors in the author’s thinking (equating diversity with gut health).
  3. A pair of well-conducted clinical trials that did not show impact of probiotics on pediatric acute diarrhea led to some ignoring all previous evidence and concluding that no probiotics were useful for acute pediatric diarrhea. ISAPP responded about the importance of putting new evidence in the context of the totality of evidence: L. rhamnosus GG for treatment of acute pediatric diarrhea: the totality of current evidence. Also, Dr. Eamonn Quigley, an ISAPP board member, published an independent response.
  4. Pieter Cohen concluded that evidence for probiotic safety is insufficient in an article in JAMA Internal Medicine. ISAPP’s response was published in a letter to the editor, along with Cohen’s response to our letter.
  5. Responding to two papers in Cell (here and here), and accompanying media coverage that called into question probiotic safety and efficacy, ISAPP published a detailed post Clinical evidence and not microbiota outcomes drive value of probiotics objecting to conclusions, and released a public statement.
  6. Jennifer Abbasi wrote a critical article about probiotics with the inflammatory title “Are Probiotics Money Down the Toilet? Or Worse?” ISAPP responded with the following blog post: Probiotics: Money Well-Spent For Some Indications.
  7. When Rao, et al incriminated probiotics as a cause of D-lactic acidosis, ISAPP posted a blog and published a letter to the editor of Clin Transl Gastroenterol objecting to this conclusion.
  8. ISAPP responded to a paper claiming that probiotics were unsafe in children: Probiotics and D-lactic acid acidosis in children and Brain Fogginess and D-Lactic Acidosis: Probiotics Are Not the Cause.

Board member and Professor Colin Hill wrote a blog post called Another day, another negative headline about probiotics? His post provides some useful questions to consider when reacting to a publication:

  • Is the article describing an original piece of research and was it published in a reputable, peer-reviewed journal?
  • What evidence is there that the strain or strain mix in question is actually a probiotic? Does it fit the very clear probiotic definition?
  • Was the study a registered human trial? How many subjects were involved? Was it blinded and conducted to a high standard?
  • What evidence was presented of the dose administered and was the strain still viable at the time of administration?
  • Were the end points of the study clear and measurable? Are they biologically or clinically significant to the subjects?
  • Did the authors actually use the words contained in the headline? “Useless”, or “waste of money”, etc?

 

Lactobacillus bacteremia in critically ill patients does not raise questions about safety for general consumers

By Dan Merenstein MD, Professor of Family Medicine, Georgetown University Medical Center, Washington DC, USA; Eamonn Quigley MD, Professor of Medicine, Houston Methodist Hospital and Weill Cornell Medical College, Texas USA; Gregory Gloor PhD, Professor of Biochemistry, Schulich School of Medicine and Dentistry, The University of Western Ontario, Canada; Hania Szajewska MD, Professor of Paediatrics, The Medical University of Warsaw, Poland;  and Mary Ellen Sanders PhD, Executive Science Officer, ISAPP, Colorado, USA.

A recent Nature Medicine paper reported blood cultures positive for L. rhamnosus GG in six critically ill patients at Children’s Hospital in Boston.

About this study

Patients (aged 1, 2, 4, 12, 19 and 19 years) with L. rhamnosus-associated bacteremia suffered from different chronic conditions (mitochondrial disorder, cerebral palsy, congenital heart disease, cystic fibrosis) and were located either in the ICU (cardiac or medical/surgical ICU) or ICP (intermediate care program) at the hospital at the time of bacteremia. The bacteremia was discovered during routine blood culture screens. Clinical presentations were not described in detail; however, none had endocarditis or died from the bacteremia, although one did get a central line infection. Two of the six cases of bacteremia in probiotic-consuming patients were determined by attending physicians to be transient or due to contaminants, and were not treated. The other four cases were treated with antibiotics. A further 516 patients dosed with the same probiotic did not develop bacteremia.

The researchers examined the blood isolates and using whole genome sequencing were able to confirm that the Lactobacillus isolated from the blood of these patients was genetically identical – with the exception of a few SNPs – to L. rhamnosus GG present in the probiotic product. This is the preferred approach to confirming the source of blood culture isolates.

Important questions arising from critical review of this paper

  1. Was the study appropriately controlled?

The authors report a seemingly high rate (1.1%) of Lactobacillus bacteremia among the 522 L. rhamnosus-consuming patients compared with 0.009%, the rate of Lactobacillus bacteremia among 21,652 patients who did not receive probiotics.  However, the paper does not justify the legitimacy of comparing these two groups to each other. Indeed, other underlying factors could contribute to the different rates of bacteremia, as these were not matched cohorts. It is important to recognize the limitations of the retrospective design used here, which limits the ability to match controls, and to control for cofounders such as underlying illness, severity of clinical illness and co-therapies (including antibiotics).

  1. What is the mechanism of transmission of the probiotic to the patients’ blood?

Most of the patients had a central line venous catheter. The paper reported that probiotics were mostly administered via tube feeding. If a probiotic is able to readily translocate the gut barrier in such patients, this would be a safety concern. But if the observed bacteremia was due to contamination of a central line, this may say more about hospital procedures than safety of the probiotic. Indeed, 16 years ago, central line contamination leading to fungemia was reported. In a 2005 paper, 92% of cases of fungemia associated with Saccharomyces cerevisiae var boulardii administration had an IV catheter. Based on such reports, handling dried probiotics in a hospital environment with critically ill patients should be done with caution. However, with proper administration procedures, certain probiotics are medically recommended in this setting.

  1. What was the clinical impact of administration of L. rhamnosus GG?

Important clinical parameters such as all-cause mortality (the outcome of greatest importance), length of hospital stay, abscesses, required medications, and others were not reported (although central line infection was reported) for the patients studied. The clinical context of this study would be more easily understood if information on the indications driving probiotic administration was provided. The authors question the risk/benefit of probiotic administration to ICU patients in a children’s hospital yet focus solely on risk and do not measure benefit. This suggests an underlying assumption by the authors that when it comes to probiotics, any risk is too much. Did the patients given L. rhamnosus GG suffer negative clinical outcomes more often than age and condition-matched controls? If so, then giving this probiotic to these patients cannot be recommended. But if not, then even though risk of bacteremia may be higher, if the patients given the probiotic fared better than matched patients, then the probiotic should be considered a reasonable option.

Lastly, the finding of a rate of Lactobacillus bacteremia of 1.1% needs to be viewed in the context of a 20% rate of nosocomial infections in the ICU (here and here).

Lessons regarding probiotic safety

Two main issues are raised by this study. The first is whether the evidence suggests opportunistic pathogenic properties of L. rhamnosus GG or rather that procedures used to administer probiotics in the ICU environment resulted in contamination, which caused bacteremia. No conclusions can be made from this study regarding this. The second is the importance of placing the results of this study into a clinical framework. The study implies risk from probiotic administration, even though the study was not powered for clinical outcomes and could not place any perceived increased risk into the context of any achieved benefit. Further, reporting rates of Lactobacillus bacteremia between cohorts unmatched for important characteristics except for probiotic use does not inform on relative risk.

Importantly for the broader situation of probiotic use, the ICU population is not reflective of the general population, so this study does not allow us to draw conclusions about safety of L. rhamnosus GG use in non-ICU patients.

We recognize the value of careful tracking of potential probiotic-associated infections and appreciate the application of bacterial genomic sequencing to identify the probiotic in the blood. Used more widely, this approach could resolve many purported claims of probiotic bacteremia.

This paper serves as an important reminder that use of probiotics in critically ill patients must be carefully considered and practice must align with learnings from the past, including the risk of central line contamination with probiotics. In addition, this paper highlights the importance of knowing the exact strain (including its antibiotic resistance profile and preferentially also its genome sequence), so that in the rare case of bacteremia, appropriate antibiotics can be administered.

See related article: Hill, C. Balancing the risks and rewards of live biotherapeuticsNat Rev Gastroenterol Hepatol (2019)

See here for an additional related open-access publication: Probiotic use in at-risk populations. Sanders et al. 2014.