Your panel
Paste a panel or load the example.
Result
Event study
estimate and 95% interval simultaneous band
Average outcome by cohort
Overall effect
| Aggregation | Estimate | SE | 95% CI |
|---|
By cohort
| First treated | Units | Effect | SE |
|---|
What the two-way fixed effects regression averages
| Comparison | Weight | Average 2x2 |
|---|
vs never treated earlier vs later later vs already treated
Every group-time effect
| Cohort | Period | Since treatment | ATT | SE | Treated | Controls |
|---|
Method and citation
Units first treated in period g form cohort g. For each cohort and period t, the group-time effect ATT(g,t) is the change in the cohort's average outcome from a base period to t, minus the same change for the comparison group (Callaway & Sant'Anna 2021). After treatment the base is the last period before it (moved earlier by the anticipation window); before treatment it is the previous period, or the same last untreated period with the universal option. The comparison group is either units never treated in the data, or those plus units not yet treated by t. Each estimate is a 2x2 difference-in-differences in the sense of Card & Krueger (1994), made only between units it is fair to compare.
Standard errors come from each estimate's influence function, clustered by unit. The overall effect averages post-treatment ATT(g,t) with weights proportional to cohort size; the event study averages those with the same time since treatment; the cohort effects average each cohort's post-treatment periods. Their standard errors include the extra term for estimating the cohort shares, as in the paper. The simultaneous band uses a multiplier bootstrap (Mammen weights, 999 draws, fixed seed) for the critical value of the largest standardised deviation across event times. The pre-test is a Wald test that every pre-treatment ATT(g,t) is zero; when their covariance is singular, as R's did also finds, the test uses the pre-treatment event-time averages instead and says so.
The two-way fixed effects estimate regresses the outcome on the treatment dummy with unit and period effects, with unit-clustered standard errors (the small-sample factor of R's fixest). Goodman-Bacon (2021) showed that in a balanced panel it equals a weighted average of every 2x2 comparison: each cohort against never-treated units, earlier against later cohorts before the later ones start, and later cohorts against already-treated ones. The weights grow with group size and with how close to the middle of the panel treatment starts. The page lists every comparison and checks that the weights sum to one and rebuild the coefficient.
Verified by tests in the source against csdid 0.4.5 (MIT), a Python port of R's did package by the method's authors, and pyfixest 0.70.0 (MIT): on the castle-doctrine example and a simulated panel, every ATT(g,t), standard error, aggregation and pre-test statistic agrees within 1e-9, for never-treated and not-yet-treated comparisons and both base-period choices, and the two-way fixed effects coefficient and clustered standard error agree within 1e-9. Planted-effect panels recover the planted effect exactly, and the decomposition rebuilds the regression coefficient on 25 random panels.
Limits
- Parallel trends, unconditionally. Without covariates, the comparison group must be on the same path the treated cohorts would have followed. Covariate-adjusted (doubly robust) estimation is not offered yet.
- Treatment stays on. Units that switch treatment off are refused; designs with reversals need other estimators.
- Balanced panels. Units missing a period are dropped, with a note.
- Small cohorts. A one-unit cohort's standard errors are too small, because its own noise cannot be estimated; inference with few treated or few control units is rough.
- No spillovers. Comparison units must not be affected by the treated units' change.
BibTeX
@article{callaway2021difference,
author = {Callaway, Brantly and Sant'Anna, Pedro H. C.},
title = {Difference-in-Differences with Multiple Time Periods},
journal = {Journal of Econometrics},
volume = {225}, number = {2}, pages = {200--230}, year = {2021},
doi = {10.1016/j.jeconom.2020.12.001}
}
@article{goodmanbacon2021difference,
author = {Goodman-Bacon, Andrew},
title = {Difference-in-Differences with Variation in Treatment Timing},
journal = {Journal of Econometrics},
volume = {225}, number = {2}, pages = {254--277}, year = {2021},
doi = {10.1016/j.jeconom.2021.03.014}
}
@article{card1994minimum,
author = {Card, David and Krueger, Alan B.},
title = {Minimum Wages and Employment: A Case Study of the Fast-Food Industry in {New Jersey} and {Pennsylvania}},
journal = {American Economic Review},
volume = {84}, number = {4}, pages = {772--793}, year = {1994}
}
@article{cheng2013castle,
author = {Cheng, Cheng and Hoekstra, Mark},
title = {Does Strengthening Self-Defense Law Deter Crime or Escalate Violence? Evidence from Expansions to Castle Doctrine},
journal = {Journal of Human Resources},
volume = {48}, number = {3}, pages = {821--854}, year = {2013},
doi = {10.3368/jhr.48.3.821}
}
Example data: log homicides per 100,000 people for the 50 US states, 2000-2010, from the FBI Uniform Crime Reports, with the first full year each state's castle-doctrine law was in force, as compiled by Cheng & Hoekstra (2013) and redistributed in the MIT-licensed causaldata Python package; provenance in data/SOURCE.md. States are numbered as in that file. The example uses no covariates, weights or region-by-year effects, so it is a teaching example, not a replication of the paper.
Questions
What does difference-in-differences answer?
Whether units that adopted a policy or change (states, stores, schools, customers) moved differently afterwards from units that had not, beyond the trend both groups shared. The effect is the change in the treated group minus the change in the comparison group over the same periods.
What is wrong with the usual two-way fixed effects regression?
When units are treated at different times, the regression of the outcome on a treatment dummy with unit and period effects is a weighted average of every two-group, two-period comparison, including comparisons that use already-treated units as controls. If the effect grows or fades over time, those comparisons subtract part of the effect, and the regression can even get the sign wrong. The Goodman-Bacon decomposition on this page shows how much weight those comparisons carry in your data.
What does the Callaway and Sant'Anna estimator do instead?
It estimates a separate effect for every cohort (the units first treated in the same period) in every period, each from a clean comparison with never-treated or not-yet-treated units, and only then averages them with weights you can see: overall, by time since treatment (the event study) or by cohort.
What data do I need?
A long, balanced panel: one row per unit and period, a numeric period column, the outcome, and either the first period each unit was treated (0 or blank for never) or a 0/1 treatment column that switches on and stays on. Units missing a period are dropped. Several periods before treatment make the pre-trends check meaningful.
How do I check parallel trends?
Look at the pre-treatment points of the event study: if the treated cohorts were already drifting away from the comparison group before treatment, the estimates left of zero will not hover around zero. The joint Wald pre-test summarises them. Passing it does not prove parallel trends, and a small sample can pass it by having little power.
Why does the tool warn about one-unit cohorts?
With one unit in a cohort there is no way to estimate how much that unit moves around by chance, so the standard errors of its estimates reflect only the comparison group and are too small. The pre-test is not reported in that case. Setting the minimum cohort size to 2 leaves such cohorts out.
Is my data uploaded anywhere?
No. The file is read and analysed by JavaScript in this browser tab. There are no accounts, cookies or analytics vendors.