Your studies
Column names are matched loosely: tpos/tneg/cpos/cneg, events_t/n_t/events_c/n_c, mean1/sd1/n1/mean2/sd2/n2, yi/sei, yi/vi or estimate/lower/upper.
Result
Forest plot
Every estimator side by side
Funnel plot
Method and citation
Effect sizes. From counts: the log risk ratio, log odds ratio or risk difference with their large-sample variances; a table with a zero cell gets 0.5 added to every cell, and a study with no events in either group is left out of a ratio analysis. From means: the mean difference, or Hedges' g with the small-sample correction J = 1 - 3/(4df - 1) (Hedges 1981) and the variance in Borenstein et al. (2009, ch. 4). From an estimate and interval: the standard error is the interval's width over 2 × 1.96, on the log scale for ratios.
Pooling. Each study is weighted by 1/(vi + τ²). The between-study variance τ² is estimated by restricted maximum likelihood (Fisher scoring; Viechtbauer 2005), the DerSimonian & Laird (1986) method of moments, or Paule & Mandel (1982); τ² = 0 gives the common-effect model. The Wald interval uses the normal quantile; the Knapp-Hartung interval (Knapp & Hartung 2003; also called Hartung-Knapp-Sidik-Jonkman) scales the variance by the weighted residual mean square and uses t on k - 1 df.
Heterogeneity. Cochran's Q; I² = τ²/(τ² + s²) with s² the typical within-study variance (Higgins & Thompson 2002; for DL this equals (Q - df)/Q); the Q-profile interval for τ², carried to I² (Viechtbauer 2007). Prediction interval: estimate ± tk-2 √(τ² + SE²) (Riley, Higgins & Deeks 2011). Small-study effects: Egger's regression of y/SE on 1/SE, intercept tested with t on k - 2 df (Egger et al. 1997). Meta-regression: a mixed-effects model with one moderator, τ² by the chosen estimator, R² as the share of τ² explained.
Checked. On the bundled trials every estimate agrees with statsmodels 0.15.0 (DerSimonian-Laird, Paule-Mandel, Hartung-Knapp, Q) and with REML found by a separate likelihood maximisation in scipy, to 1e-6 or better, and with the analysis of the same trials printed in metafor's README to every digit shown (τ² 0.3132, I² 92.22%, Hartung-Knapp interval -1.1084 to -0.3206 on the log scale). In 2000 simulated meta-analyses for each of six settings (5 to 20 studies), the 95% Wald interval covered the true mean 87 to 94% of the time and the Hartung-Knapp interval 93 to 96%; the prediction interval held 85 to 93% of the true study effects.
@article{dersimonian1986meta, author = {DerSimonian, Rebecca and Laird, Nan}, title = {Meta-analysis in clinical trials}, journal = {Controlled Clinical Trials}, volume = {7}, number = {3}, pages = {177--188}, year = {1986}, doi = {10.1016/0197-2456(86)90046-2}}
@article{viechtbauer2005bias, author = {Viechtbauer, Wolfgang}, title = {Bias and efficiency of meta-analytic variance estimators in the random-effects model}, journal = {Journal of Educational and Behavioral Statistics}, volume = {30}, number = {3}, pages = {261--293}, year = {2005}, doi = {10.3102/10769986030003261}}
@article{paule1982consensus, author = {Paule, Robert C. and Mandel, John}, title = {Consensus values and weighting factors}, journal = {Journal of Research of the National Bureau of Standards}, volume = {87}, number = {5}, pages = {377--385}, year = {1982}, doi = {10.6028/jres.087.022}}
@article{knapp2003improved, author = {Knapp, Guido and Hartung, Joachim}, title = {Improved tests for a random effects meta-regression with a single covariate}, journal = {Statistics in Medicine}, volume = {22}, number = {17}, pages = {2693--2710}, year = {2003}, doi = {10.1002/sim.1482}}
@article{higgins2002quantifying, author = {Higgins, Julian P. T. and Thompson, Simon G.}, title = {Quantifying heterogeneity in a meta-analysis}, journal = {Statistics in Medicine}, volume = {21}, number = {11}, pages = {1539--1558}, year = {2002}, doi = {10.1002/sim.1186}}
@article{riley2011interpretation, author = {Riley, Richard D. and Higgins, Julian P. T. and Deeks, Jonathan J.}, title = {Interpretation of random effects meta-analyses}, journal = {BMJ}, volume = {342}, pages = {d549}, year = {2011}, doi = {10.1136/bmj.d549}}
@article{viechtbauer2007confidence, author = {Viechtbauer, Wolfgang}, title = {Confidence intervals for the amount of heterogeneity in meta-analysis}, journal = {Statistics in Medicine}, volume = {26}, number = {1}, pages = {37--52}, year = {2007}, doi = {10.1002/sim.2514}}
@article{egger1997bias, author = {Egger, Matthias and Davey Smith, George and Schneider, Martin and Minder, Christoph}, title = {Bias in meta-analysis detected by a simple, graphical test}, journal = {BMJ}, volume = {315}, number = {7109}, pages = {629--634}, year = {1997}, doi = {10.1136/bmj.315.7109.629}}
@article{hedges1981distribution, author = {Hedges, Larry V.}, title = {Distribution theory for Glass's estimator of effect size and related estimators}, journal = {Journal of Educational Statistics}, volume = {6}, number = {2}, pages = {107--128}, year = {1981}, doi = {10.3102/10769986006002107}}
@article{colditz1994efficacy, author = {Colditz, Graham A. and Brewer, Timothy F. and Berkey, Catherine S. and Wilson, Mary E. and Burdick, Elisabeth and Fineberg, Harvey V. and Mosteller, Frederick}, title = {Efficacy of {BCG} vaccine in the prevention of tuberculosis: meta-analysis of the published literature}, journal = {JAMA}, volume = {271}, number = {9}, pages = {698--702}, year = {1994}, doi = {10.1001/jama.1994.03510330076038}}
Limits
- Garbage in. A pooled estimate is only as good as the studies and how they were found. This tool does not search, screen or judge risk of bias; a systematic review does, and its protocol matters more than the choice of estimator.
- Few studies. With fewer than about five studies τ² is barely estimable, I² can swing from 0 to 90% with one study, and the Wald interval is too narrow. Read the Hartung-Knapp interval and the I² interval.
- Large-sample variances. The variances of log ratios and Hedges' g are approximations that need a fair number of events and participants per study. Rare events, many zero cells or tiny trials call for exact or one-stage models (for example the Mantel-Haenszel or Peto methods, or a binomial-normal model), which are not here.
- Moderators. A meta-regression compares studies, not people: a slope across studies can differ from the effect within any of them (ecological bias), and with fewer than 10 studies per moderator it is exploratory.
- Not modelled: dependent effect sizes from one study (multilevel models), network meta-analysis, individual participant data, trim-and-fill and selection models for publication bias, subgroup tests by a categorical column, and Bayesian estimators.
Questions
Fixed effect or random effects?
A common-effect (fixed) model assumes every study measures the same true effect and differs only by chance. A random-effects model lets the true effect vary between studies and estimates the mean of that distribution. When studies differ in population, dose or setting, the random-effects model is the honest default; the two agree when heterogeneity is zero.
What does I-squared mean?
The share of the spread in study results that reflects real differences between studies rather than chance. It is not the size of those differences: tau, the standard deviation of the true effects, is. With few studies I-squared is very uncertain, which is why its confidence interval is shown.
Why show a prediction interval?
The confidence interval says where the average effect is. The prediction interval says where the true effect of a new study, in a new setting, is likely to fall. With large heterogeneity a treatment can help on average and still do nothing, or harm, in some settings; the prediction interval shows that and the confidence interval hides it.
Should I use the Hartung-Knapp interval?
With few studies, usually yes. The usual Wald interval treats the between-study variance as known. In simulations on this page the 95% Wald interval covered the true mean 87 to 94% of the time, and the Hartung-Knapp interval 93 to 96%.
Does a significant Egger's test prove publication bias?
No. It says small studies report different effects from large ones. Publication bias is one cause; others are real differences in small studies (different patients, more intensive treatment), poorer methods in small studies, and chance. With fewer than 10 studies the test has little power.