A Miracle Treatment! Or Not?
By Daniel J. Merenstein, MD, Professor, Department of Family Medicine and Director of Research Programs, Georgetown University Medical Center, Washington DC
Here’s a scenario for a physician: A drug rep walks into your office. She has a new product she wants to talk to you about. You are super excited to talk to her as you have heard all about this product from many other sources. The data that are being reported are amazing. There are hundreds if not many more case reports of it working. People were dying and then totally recovered after being given this product. It has been witnessed and published! The efficacy is well over 90%. You are not sure there is any intervention you have ever heard of that has such amazing efficacy. She tells you that in some of the cases, the patients were very sick and despite numerous courses of antibiotics they did not improve until this new product was given. You ask for more information as you are starting to think this must be like when doctors first heard of penicillin.
The product can be taken orally but that is not the way it is generally given. She tells you that although there are 2-3 ways to administer, most hospitals are doing it the most expensive way now. (You later learn that the typical– and most expensive – approach to administering the product may not even be the best approach.) But you withhold judgement as this sounds exciting. And remember, you have been hearing all about this from so many different sources.
But as you listen, it gets a little confusing. She tells you that the makeup of the product is different in nearly every application. This makes it exciting to use, as one really never knows what is in it. It is also relatively cheap to obtain, as the patient can have a friend just bring it in for them.
Since you are trained in evidence-based medicine, you ask a few questions. It is exciting there are all sorts of case reports but what about the randomized controlled trials, and what does the FDA say about it? You ask if you can look at the trials—there is no way you can review hundreds of studies now but if she leaves them for you, you will look at them this weekend. But before she leaves you ask a few quick questions. How many of these studies are randomized? She says 10. How many use a placebo? She says 6. You tell her what you really want to do is review all the randomized placebo-controlled blinded studies, if she can just leave those.
Later in the week you go pick up the folder she left and right away are a little surprised at how light it is. It looks like there are only 3 randomized placebo-controlled blinded studies, only two of which are peer-reviewed and published. One was a positive study; overall, 91% of patients in the new drug group achieved clinical cure compared with 63% in the control group. But you realize this is not exactly a placebo-controlled trial. What they did is compare two types of the new application. Furthermore, this study was conducted at two sites and at one of the sites both the new application and the control had nearly identical rates of improvement, both over 90%. Okay so this was not a perfect study, only 46 total participants, but still pretty exciting with over 90% improvement.
The second study had three groups of 83 people. Group A (2 doses of new drug), B (2 doses of placebo) and C (1 dose of new drug and 1 placebo dose). The efficacy for these three groups was 61%, 45%, and 67%, respectively. The primary endpoint was not met (P = .152). Interestingly, Group C, which included one dose of placebo, was superior to all placebo (group B) but Group A, in which the drug was given two times, was not superior to placebo.
The third study, a Phase II trial, appears to not be peer-reviewed or published, but just reported online. However, it does appear this was far from a positive study, with 44% of subjects (26 of 59) who received the new application improving versus 53% of subjects (16 of 30) who received placebo. I have been told that this study will be published soon and that a Phase III study of this intervention was also undertaken.
Well now you are getting a little more confused. You have heard from fellow docs, the lay press, medical literature and the drug rep that this new application was over 90% effective. But it appears in the three reasonably well controlled studies, the ones from which we can really draw conclusions, only one was positive and in that study the control was not a real placebo.
Besides efficacy, you remember that one has to always consider the cost and adverse events. Maybe this new application is like recommending the Mediterranean Diet, where the efficacy from studies is limited but the adverse events are nearly non-existent. But when you do a quick PubMed search you learn that this is far from the case with this product. This application has been reported to cause very serious adverse events, including extended-spectrum beta-lactamase (ESBL)–producing Escherichia coli bacteremia resulting in one death. You look online expecting that the FDA must have some serious warnings about this new drug. You don’t find any such warnings.
You may have guessed that the product is in fact a Fecal Microbiota Transplant (FMT). Besides having a professional interest in this much-discussed treatment, I have a personal interest. Last year my son was in a Johns Hopkins Hospital with a central line and two broad-spectrum antibiotics for a bone infection. I asked them to provide him with probiotics since the number needed to treat to prevent pediatric antibiotic associated diarrhea is 9, per a 2019 Cochrane review. This review included 20 randomized, placebo-controlled studies of a single strain. However, I was told no Hopkins hospital will administer probiotics, and further, that we could not even bring in our own because of concerns for the safety of others. But no worries – if my son got recurrent C. diff infection, Hopkins would allow this great new procedure, FMT.
In medicine I cannot truly imagine a probiotic with the same evidence base as FMT receiving such widespread acceptance and escaping regulatory scrutiny. And currently used probiotics have an excellent safety record. Just imagine, if this were a new drug being sold there would be widespread condemnation of the attempt to get approval mainly based on anecdotal case reports. Shockingly, based on the level of evidence I have described many experts now think a randomized placebo-controlled trial is not even ethical for the placebo group, as of course they know FMT works.
It is a quandary. I am not opposed to FMT; I find it fascinating. But why has it been so widely accepted and why has the FDA, which in general has been very careful with probiotic applications in medicine, allowed this to proceed for recurrent C diff infection with only enforcement discretion? Both treatments administer live microorganisms, one with 31 placebo controlled randomized trials, including 8672 subjects [of C. diff prevention (number needed to prevent=42), not treatment like FMT], the other with pretty limited data. I have my thoughts, but better for you to ponder it.
Additional related content:
Webinar presenting current level of evidence for FMT: FECAL MICROBIOTA TRANSPLANTATION, AM I SURE IT WORKS? Oct 29, 2020. Presented by Prof. Daniel Merenstein, introduced by Prof. Hania Szajewska, sponsored by Centro Studi Scientifici, La Marcigliana.
