A new study by Ignacio Atal, long-term researcher at CRI Research, and collaborators at Hôpital Hôtel-Dieu, has just been published in the Journal of Clinical Epidemiology. It shows how the statistical significance of meta-analyses can be fragile if single events in the studies included in the meta-analysis are altered.
The full text of the study is available here
Meta-analyses inform clinical practice by summarizing the effect of a treatment from the results of several trials. However, the statistical significance of such a meta-analysis may rely on the results of only a few patients from the specific trials included. The study assesses this effect by defining the meta-analysis fragility index, simply a minimum number of changes in patient outcome (e.g. cured or not cured) that would alter the study's conclusions. Among the 906 meta-analyses reviewed, the median fragility index was 12, but for around 30 % of the studies, it was 5 or less. This indicates an overall fragility of the meta-analysis results in the face of small changes in the specific studies considered.
The authors have also created a website which allows other scientists to assess the fragility of their meta-analyses.
