Improving the quality of microbiome studies – STORMS

By Mary Ellen Sanders, PhD, ISAPP Executive Science Officer

In mid-March I attended the Gut Microbiota for Health annual meeting. I was fortunate to participate in a short workshop chaired by Dr. Geoff Preidis MD, PhD, a pediatric gastroenterologist from Baylor College of Medicine and Dr. Brendan Kelly MD, MSCE, an infectious disease physician and clinical epidemiologist from University of Pennsylvania. The topic of this workshop was “Designing microbiome trials – unique considerations.”

Dr. Preidis introduced the topic by recounting his effort (Preidis et al. 2020) to review evidence for probiotics for GI endpoints, including for his special interest, necrotizing enterocolitis (NEC). After a thorough review of available studies testing the ability of probiotics to prevent morbidity and mortality outcomes for premature neonates, he and the team found 63 randomized controlled trials that assessed close to 16,000 premature babies. Although the effect size for the different clinical endpoints was impressive and clinically meaningful, AGA was only able to give a conditional recommendation for probiotic use in this population.

Why? In part, because inadequate conduct or reporting of these studies led to reduced confidence in their conclusions. For example, proper approaches to mitigate selection bias must be reported. Some examples of selection bias include survival bias (where part of the target study population is more likely to die before they can be studied), convenience sampling (where members of the target study population are not selected at random), and loss to follow-up (when probability of dropping out is related to one of the factors being studied). These are important considerations that might influence microbiome results. If the publication on the trial does not clearly indicate how these potential biases were addressed, then the study cannot be judged as low risk of bias. It’s possible in such a study that bias is addressed correctly but reported incompletely. But the reader cannot ascertain this.

With an eye toward improving the quality and transparency of future studies that include microbiome endpoints, Dr. Preidis shared a paper by a multidisciplinary team of bioinformaticians, epidemiologists, biostatisticians, and microbiologists titled Strengthening The Organization and Reporting of Microbiome Studies (STORMS): A Reporting Checklist for Human Microbiome Research.

Dr. Preidis kindly agree to help the ISAPP community by answering a few questions about STORMS:

Dr. Preidis, why is the STORMS approach so important?

Before STORMS, we lacked consistent recommendations for how methods and results of human microbiome research should be reported. Part of the problem was the complex, multi-disciplinary nature of these studies (e.g., epidemiology, microbiology, genomics, bioinformatics). Inconsistent reporting negatively impacts the field because it renders studies difficult to replicate or compare to similar studies. STORMS is an important step toward gaining more useful information from human microbiome research.

One very practical outcome of this paper is a STORMS checklist, which is intended to help authors provide a complete and concise description of their study. How can we get journal editors and reviewers to request this checklist be submitted along with manuscripts for publication?

We can reach out to colleagues who serve on editorial boards to initiate discussions among the editors regarding how the STORMS checklist might benefit reviewers and readers of a specific journal.

How does this checklist differ from or augment the well-known CONSORT checklist?

Whereas the CONSORT checklist presents an evidence-based, minimum set of recommendations for reporting randomized trials, the STORMS checklist facilitates the reporting of a comprehensive array of observational and experimental study designs including cross-sectional, case-control, cohort studies, and randomized controlled trials. In addition to standard elements of study design, the STORMS checklist also addresses critical components that are unique to microbiome studies. These include details on the collection, handling, and preservation of specimens; laboratory efforts to mitigate batch effects; bioinformatics processing; handling of sparse, unusually distributed multi-dimensional data; and reporting of results containing very large numbers of microbial features.

How will papers reported using STORMS facilitate subsequent meta-analyses?

When included as a supplemental table to a manuscript, the STORMS checklist will facilitate comparative analysis of published results by ensuring that all key elements are reported completely and organized in a way that makes the work of systematic reviewers more efficient and more accurate.

I have been struck through the years of reading microbiome research that primary and secondary outcomes seem to be rarely stated up front. Or if such trials are registered, for example on, the paper does not necessarily focus on the pre-stated primary objectives. This approach runs the risk of researchers finding the one positive story to tell out of the plethora of data generated in microbiome studies. Will STORMS help researchers design more hypothesis driven studies?

Not necessarily. The STORMS checklist was not created to assess study or methodological rigor; rather, it aims to aid authors’ organization and ease the process of reviewer and reader assessment of how studies are conducted and analyzed.  However, if investigators use this checklist in the planning phases of a study in conjunction with sound principles of study design, I believe it can help improve the quality of human microbiome studies – not just the writing and reporting of results.

Do you have any additional comments?

One of the strengths of the STORMS checklist is that it was developed by a multi-disciplinary team representing a consensus across a broad cross-section of the microbiome research community. Importantly, it remains a work in progress, with planned updates that will address evolving standards and technological processes.  Anyone interested in joining the STORMS Consortium should visit the consortium website (

See related blog:   ISAPP take-home points from American Gastroenterological Association guidelines on probiotic use for gastrointestinal disorders


A postbiotic is not simply a dead probiotic

By Dr. Gabriel Vinderola, PhD,  Associate Professor of Microbiology at the Faculty of Chemical Engineering from the National University of Litoral and Principal Researcher from CONICET at Dairy Products Institute (CONICET-UNL), Santa Fe, Argentina

Postbiotics, recently addressed in an ISAPP consensus panel paper, are defined as a preparation of inanimate microorganisms and/or their components that confers a health benefit on the host. Criteria to meet the postbiotic definition are summarized here. One noteworthy aspect of this definition is that the word ‘probiotic’ does not appear. Although in practice a probiotic strain may be used as a progenitor strain in the manufacture of a postbiotic, the simple process of inactivating a probiotic is not sufficient to be called a postbiotic. It cannot be assumed that any non-viable probiotic cells in a probiotic product are automatically considered a postbiotic component. If a probiotic strain is used as a progenitor of a postbiotic, an efficacy study must be redone using the inanimate preparation and a benefit must be demonstrated. A probiotic product displaying fewer than the labeled count of viable cells is merely a low-quality product; it is not a postbiotic.

Further, the ISAPP consensus definition on postbiotics recognizes that the process of making a postbiotic implies a deliberate step to inactivate the viable cells of the progenitor strain. This process can be achieved by different technological steps such as heat-treatment (perhaps the most feasible approach), high pressure, radiation or simply aerobic exposure for strict anaerobes. A corresponding efficacy study must be conducted on the preparation. Or at the very least, any postbiotic component of a probiotic product must be specifically shown to contribute to the health benefit conferred by the product.

In contrast to postbiotics, probiotics are live microorganisms which when administered in adequate amounts confer a health benefit on the host. Four minimum criteria should be met for a strain to be considered as a probiotic: (i) sufficiently characterized; (ii) safe for the intended use; (iii) supported by at least one positive human clinical trial conducted according to generally accepted scientific standards or as per recommendations and provisions of local/national authorities when applicable; and (iv) alive in the product at an efficacious dose throughout shelf life (Binda et al. 2020). This last requirement reflects the key difference between probiotics and postbiotics. Probiotics must deliver an efficacious number of viable cells through the shelf life of the product. In practice, probiotic products may display significant numbers of non-viable cells (Raymond & Champagne, 2015), as some cells may lose viability during the technological process of biomass production, while undergoing manufacture or preservation steps and through product storage prior to purchase. In order to provide the target dose until end of shelf life, an overage of 0.5 to 1 log order CFU above the expected counts of viable cells is commonly included in the product to compensate for potential losses during product storage and handling (Fenster et al. 2019).

Thus, some quantity of non-viable cells may be usually expected in certain probiotic products, especially supplement products claiming a long, room temperature stable shelf-life. However, they will be considered as probiotic products of quality as long as they are able to deliver the expected amount of viable cells until the end of the product shelf-life. It is worth mentioning that the probiotics are expected to be viable at the moment of their administration. After that, if exposure to different regions of the gut causes cells to die, it is not of consequence as long as a health effect is achieved.

Probiotics and postbiotics have things in common (the need of efficacy studies that demonstrates their benefits) and things that distinguish them (the former are administered alive, whereas the latter are administrated in their inanimate form), but no probiotic becomes a postbiotic just by losing cell viability during storage.

Follow up from ISAPP webinar – Probiotics, prebiotics, synbiotics, postbiotics and fermented foods: how to implement ISAPP consensus definitions

By Mary Ellen Sanders PhD, Executive Science Officer, ISAPP

On the heels of the most recent ISAPP consensus paper – this one on postbiotics – ISAPP sponsored a webinar for industry members titled Probiotics, prebiotics, synbiotics, postbiotics and fermented foods: how to implement ISAPP consensus definitions. This webinar featured short presentations outlining definitions and key attributes of these five substances. Ample time remained for the 10 ISAPP board members to field questions from attendees.

When considering the definitions, it’s important to remember that the definition is a starting point – not all criteria can be included. Using the probiotic definition as an example, Prof. Colin Hill noted that the definition is only 15 words – Live microorganisms that, when administered in adequate amounts, confer a health benefit on the host. This is a useful definition, stipulating the core characteristics of a probiotic. However, important criteria such as safety and identity are not specified in the definition yet are clearly delineated in the full paper on probiotics.

Several interesting topics emerged from this discussion, which will be explored in future blog posts. These include:

  • What is meant by host health? Microbe mediated benefits are numerous. But not all benefits are a benefit to host health. Benefits for user appearance, pets and potentially livestock may be measurable, economically important and desirable, but may not encompass ‘host health’.
  • What types of endpoints are appropriate for studies to meet the requirement of a health benefit? Endpoints that indicate improved health (such as symptom alleviation, reduced incidence of infections or quality of life measures) are targeted. Some physiological measures that may be linked to health (such as increased fecal short chain fatty acids or changes in microbiota composition) may not be sufficient.
  • What are the regulatory implications from these definitions? As suggested by the National Law Review article on the ISAPP consensus definitions, attorneys are interested in the scientific positions on how these terms are defined and characterized. Further, some regulatory actions – such as by Codex Alimentarius in defining probiotics – are underway. ISAPP is open to suggestions about the best way to communicate these definitions to regulators.
  • Is any follow-up by ISAPP to these papers anticipated in order to clarify criteria and provide simple guidance to their implementation?

Simple guidance to these substances can be found in the infographics: probiotics, probiotic criteria, prebiotics, fermented foodshow are probiotic foods and fermented foods different, synbiotics, and postbiotics. As mentioned above, watch for blog updates on implementation of the definitions for different stakeholder groups.

The recording of this webinar is available here under password protection for ISAPP industry members only.

Related information:

Consensus panel papers, all published in Nature Reviews Gastroenterology and Hepatology:

A roundup of the ISAPP consensus definitions: probiotics, prebiotics, synbiotics, postbiotics and fermented foods





A roundup of the ISAPP consensus definitions: probiotics, prebiotics, synbiotics, postbiotics and fermented foods

By Dr. Mary Ellen Sanders, PhD, ISAPP Executive Science Officer

ISAPP has long recognized the importance of precise definitions of the ‘biotic’ family of terms. As a scientific organization working to advance global knowledge about probiotics, prebiotics, synbiotics, postbiotics and fermented foods, we believe carrying out rigorous scientific studies—and comparing one result to another—is more difficult if we do not start with a clear definition of what we are studying.

Over the past 8 years, ISAPP has endeavored to bring clarity to these definitions for scientists and other stakeholders. ISAPP board members have met with other top experts representing multiple perspectives and specialties in the field to develop precise, useful and appropriate definitions of the terms probiotics, prebiotics, synbiotics, postbiotics and fermented foods. The definitions of these first four terms have all entailed the requirement that the substance be shown to confer a health benefit in the target host. Fermented foods have multitudes of sensorial, nutritional and technological benefits, which drive their utility. A health benefit is not required.

The problem with health benefits

The definitions provide significant advantages for the scientific community in terms of clarity but complexity arises when the same definitions are accepted by regulatory agencies. This requirement for a health benefit as part of the probiotic definition has been rigorously implemented in the European Union. Currently, with the exception of a few member states, the term probiotic is prohibited. The logic is that since a health benefit is inherent to the term probiotic and since there are no approved health claims for probiotics in the EU*, the term ‘probiotic’ is seen to be acting as a proxy for a health claim. This has frustrated probiotic product companies who believe they have met the scientific criteria for probiotics, yet cannot identify their product as a probiotic in the marketplace because they have not received endorsement of their claims by the EU. This is not an issue resulting from an unclear definition, since probiotics surely should provide a health benefit, but rather from a lack of agreement as to what level of evidence is sufficient to substantiate a health benefit.

ISAPP remains committed to the importance of requiring a health benefit for the ‘biotic’ family of terms (outlined in the table below). It’s clear that all of these definitions are meaningless unless the requirement that they confer a health benefit is considered as essential by all stakeholders. One could reasonably discuss whether the required levels of evidence for foods and supplements are too high in some regulatory jurisdictions, but the value of these substances collapses in the absence of a health benefit.

Summary of ISAPP consensus definitions

With the publication of the most recent ISAPP consensus paper, this one on postbiotics, I offer a summary below of the five consensus definitions published by ISAPP. Each definition is part of a comprehensive paper resulting from focused discussions among experts in the field and published in Nature Reviews Gastroenterology and Hepatology (NRGH). These papers are among the top most viewed of all time on the NRGH website and are increasingly cited by scientists and regulators.

Table. Summary of ISAPP Consensus Definitions of the ‘Biotics’ Family of Substances. Probiotics, prebiotics, synbiotics and postbiotics have in common the requirement for a health benefit. They may apply to any target host, any regulatory category and must be safe for their intended use. Fermented foods fall only under the foods category and no health benefit is required.

Definition Key features of the definition Reference
Probiotics Live microorganisms that, when administered in adequate amounts, confer a health benefit on the host Grammatical correction of the 2001 FAO/WHO definition.

No mechanism is stipulated by the definition.


Hill et al. 2014
Prebiotics A substrate that is selectively utilized by host microorganisms conferring a health benefit Prebiotics are distinct from fiber. Beneficial impact on resident microbiota and demonstration of health benefit required in same study. Gibson et al. 2017
Synbiotics A mixture comprising live microorganisms and substrate(s) selectively utilized by host microorganisms that confers a health benefit on the host Two types of synbiotics defined: complementary and synergistic. Complementary synbiotics comprise probiotic(s) plus prebiotic(s), meeting requirements for criteria for each. Synergistic synbiotics comprise substrate(s) selectively utilized by co-administered live microbe(s), but independently, the components do not have to meet criteria for prebiotic or probiotic. Swanson et al. 2020
Postbiotics Preparation of inanimate microorganisms and/or their components that confers a health benefit on the host Postbiotics are prepared from live microbes that undergo inactivation and the cells or cellular structures must be retained. Filtrates or isolated components from the growth of live microbes are not postbiotics. A probiotic that is killed is not automatically a postbiotic; the preparation must be shown to confer a health benefit, as well as meet all other criteria for a postbiotic. Salminen et al. 2021
Fermented Foods Foods made through desired microbial growth and enzymatic conversions of food components Fermented foods are not the same as probiotics. They are not required to have live microbes characterized to the strain level nor have evidence of a health benefit. Types of fermented foods are many and are specific to geographic regions. Compared to the raw foods they are made from, they may have improved taste, digestibility, safety, and nutritional value. Marco et al. 2021



*Actually, there is one approved health claim in the EU for a probiotic: Scientific Opinion on the substantiation of health claims related to live yoghurt cultures and improved lactose digestion (ID 1143, 2976) pursuant to Article 13(1) of Regulation (EC) No 1924/2006


Further reading: Defining emerging ‘biotics’

New publication addresses the question: Which bacteria truly qualify as probiotics?

Although the international scientific consensus definition of probiotics, published in 2014, is well known—”live microorganisms that, when administered in adequate amounts, confer a health benefit on the host”—the word is often used incorrectly in practice.

A recent article published in Frontiers in Microbiology builds on this definition and describes four criteria for accurate use of the word ‘probiotic’. Eight scientists co-authored the paper, including two ISAPP board members. The project was initiated by industry scientists affiliated with IPA Europe.

The authors explain why it’s important for scientists and companies to be sure the four identified criteria apply before using the term ‘probiotic’. Given the many misuses of the term that are evident today, however, consumers need to scrutinize ‘probiotic’ products to be sure they are legitimate.

Read the ISAPP press release on this publication here.

See an infographic summary of this publication here.



Is probiotic colonization essential?

By Prof. Maria Marco, PhD, Department of Food Science & Technology, University of California, Davis

It is increasingly appreciated by consumers, physicians, and researchers alike that the human digestive tract is colonized by trillions of bacteria and many of those bacterial colonists have important roles in promoting human health. Because of this association between the gut microbiota and health, it seems appropriate to suggest that probiotics consumed in foods, beverages, or dietary supplements should also colonize the human digestive tract. But do probiotics really colonize? What is meant by the term “colonization” in the first place? If probiotics don’t colonize, does that mean that they are ineffective? In that case, should we be searching for new probiotic strains that have colonization potential?

My answer to the first question is no – probiotics generally do not colonize the digestive tract or other sites on the human body. Before leaping to conclusions on what this means for probiotic efficacy, “colonization” as defined here means the permanent, or at least long-term (weeks, months, or years) establishment at a specific body site. Colonization can also result in engraftment with consequential changes to the gut microbiota composition and function. For colonization to occur, the probiotic should multiply and form a stably replicating population. This outcome is distinct from a more transient, short-term (a few days to a week or so) persistence of a probiotic. For transient probiotics, it has been shown in numerous ways that they are metabolically active in the intestine and might even grow and divide. However, they are not expected to replicate to high numbers or displace members of the native gut microbiota.

Although some studies have shown that digestive tracts of infants can be colonized by probiotics (weeks to months), the intestinal persistence times of probiotic strains in children and adults is generally much shorter, lasting only few days. This difference is likely due to the resident gut microbiota that develops during infancy and tends to remain relatively stable throughout adulthood. Even with perturbations caused by antibiotics or foodborne illness, the gut microbiome tends to be resilient to the long-term establishment of exogenous bacterial strains. In instances where probiotic colonization or long-term persistence was found, colonization potential has been attributed more permissive gut microbiomes specific to certain individuals. In either case, for colonization to occur, any introduced probiotic has to overcome the significant ecological constraints inherent to existing, stable ecosystems.

Photo by Copyright, ISAPP 2019.

This leads to the next question: Can probiotics confer health benefits even if they do not colonize? My answer is definitely yes! Human studies on probiotics with positive outcomes have not relied on intestinal colonization by those microbes to cause an effect. Instead of colonizing, probiotics can alter the digestive tract in other ways such as by producing metabolites that modulate the activity of the gut microbiota or stimulate the intestinal epithelium directly. These effects could happen even on short-time scales, ranging from minutes to hours.

Should we be searching for new probiotic strains that have greater colonization potential? By extension of what we know about the resident human gut microbiota, it is increasingly attractive to identify bacteria that colonize the human digestive tract in the same way. In some situations, colonization might be preferred or even essential to impacting health, such as by engrafting a microbe that performs critical metabolic functions in the gut (e.g. break down complex carbohydrates). However, colonization also comes with risks of unintended consequences and the loss of ability to control the dose, frequency, and duration of exposure to that particular microbe.

Just as most pharmaceutical drugs have a transient impact on the human body, why should we expect more from probiotics? Many medications need to be taken life-long in order manage chronic conditions. Single or even repeated doses of any medication are similarly not expected to cure disease. Therefore, we should not assume a priori that any observed variations in probiotic efficacy are due to a lack of colonization. To the contrary, the consumption of probiotics could be sufficient for a ripple effect in the intestine, subtly altering the responses of the gut microbiome and intestinal epithelium in ways that are amplified throughout the body. Instead of aiming for engraftment directly or hand-wringing due to a lack of colonization, understanding the precise molecular interactions and cause/effect consequences of probiotic introduction will lead to a path that ultimately determines whether colonization is needed or just a distraction.