DIFFERENCE-IN-DIFFERENCES · STAGGERED ADOPTION · CALLAWAY & SANT'ANNA · GOODMAN-BACON · RUNS IN YOUR BROWSER · NOTHING UPLOADED

Did the change move the outcome, when units adopted it at different times?

States pass a law in different years, stores get a new layout in different months, schools adopt a programme one cohort at a time. Upload a panel of every unit over time with the period each was first treated. You get a clean effect for every cohort and period (Callaway & Sant'Anna), an event-study plot with pointwise and simultaneous bands, a pre-trends test, the familiar two-way fixed effects estimate, and its Goodman-Bacon decomposition, which shows how much of that estimate comes from comparing late adopters with units already treated.

Nothing you enter leaves this browser tab. Statistical output from data you supply, not advice. Not legal, tax, or financial advice, and not medical or policy advice.

Your panel

Paste a panel or load the example.

Method and citation

Units first treated in period g form cohort g. For each cohort and period t, the group-time effect ATT(g,t) is the change in the cohort's average outcome from a base period to t, minus the same change for the comparison group (Callaway & Sant'Anna 2021). After treatment the base is the last period before it (moved earlier by the anticipation window); before treatment it is the previous period, or the same last untreated period with the universal option. The comparison group is either units never treated in the data, or those plus units not yet treated by t. Each estimate is a 2x2 difference-in-differences in the sense of Card & Krueger (1994), made only between units it is fair to compare.

Standard errors come from each estimate's influence function, clustered by unit. The overall effect averages post-treatment ATT(g,t) with weights proportional to cohort size; the event study averages those with the same time since treatment; the cohort effects average each cohort's post-treatment periods. Their standard errors include the extra term for estimating the cohort shares, as in the paper. The simultaneous band uses a multiplier bootstrap (Mammen weights, 999 draws, fixed seed) for the critical value of the largest standardised deviation across event times. The pre-test is a Wald test that every pre-treatment ATT(g,t) is zero; when their covariance is singular, as R's did also finds, the test uses the pre-treatment event-time averages instead and says so.

The two-way fixed effects estimate regresses the outcome on the treatment dummy with unit and period effects, with unit-clustered standard errors (the small-sample factor of R's fixest). Goodman-Bacon (2021) showed that in a balanced panel it equals a weighted average of every 2x2 comparison: each cohort against never-treated units, earlier against later cohorts before the later ones start, and later cohorts against already-treated ones. The weights grow with group size and with how close to the middle of the panel treatment starts. The page lists every comparison and checks that the weights sum to one and rebuild the coefficient.

Verified by tests in the source against csdid 0.4.5 (MIT), a Python port of R's did package by the method's authors, and pyfixest 0.70.0 (MIT): on the castle-doctrine example and a simulated panel, every ATT(g,t), standard error, aggregation and pre-test statistic agrees within 1e-9, for never-treated and not-yet-treated comparisons and both base-period choices, and the two-way fixed effects coefficient and clustered standard error agree within 1e-9. Planted-effect panels recover the planted effect exactly, and the decomposition rebuilds the regression coefficient on 25 random panels.

Limits

BibTeX

@article{callaway2021difference,
  author  = {Callaway, Brantly and Sant'Anna, Pedro H. C.},
  title   = {Difference-in-Differences with Multiple Time Periods},
  journal = {Journal of Econometrics},
  volume  = {225}, number = {2}, pages = {200--230}, year = {2021},
  doi     = {10.1016/j.jeconom.2020.12.001}
}
@article{goodmanbacon2021difference,
  author  = {Goodman-Bacon, Andrew},
  title   = {Difference-in-Differences with Variation in Treatment Timing},
  journal = {Journal of Econometrics},
  volume  = {225}, number = {2}, pages = {254--277}, year = {2021},
  doi     = {10.1016/j.jeconom.2021.03.014}
}
@article{card1994minimum,
  author  = {Card, David and Krueger, Alan B.},
  title   = {Minimum Wages and Employment: A Case Study of the Fast-Food Industry in {New Jersey} and {Pennsylvania}},
  journal = {American Economic Review},
  volume  = {84}, number = {4}, pages = {772--793}, year = {1994}
}
@article{cheng2013castle,
  author  = {Cheng, Cheng and Hoekstra, Mark},
  title   = {Does Strengthening Self-Defense Law Deter Crime or Escalate Violence? Evidence from Expansions to Castle Doctrine},
  journal = {Journal of Human Resources},
  volume  = {48}, number = {3}, pages = {821--854}, year = {2013},
  doi     = {10.3368/jhr.48.3.821}
}

Example data: log homicides per 100,000 people for the 50 US states, 2000-2010, from the FBI Uniform Crime Reports, with the first full year each state's castle-doctrine law was in force, as compiled by Cheng & Hoekstra (2013) and redistributed in the MIT-licensed causaldata Python package; provenance in data/SOURCE.md. States are numbered as in that file. The example uses no covariates, weights or region-by-year effects, so it is a teaching example, not a replication of the paper.

Questions

What does difference-in-differences answer?

Whether units that adopted a policy or change (states, stores, schools, customers) moved differently afterwards from units that had not, beyond the trend both groups shared. The effect is the change in the treated group minus the change in the comparison group over the same periods.

What is wrong with the usual two-way fixed effects regression?

When units are treated at different times, the regression of the outcome on a treatment dummy with unit and period effects is a weighted average of every two-group, two-period comparison, including comparisons that use already-treated units as controls. If the effect grows or fades over time, those comparisons subtract part of the effect, and the regression can even get the sign wrong. The Goodman-Bacon decomposition on this page shows how much weight those comparisons carry in your data.

What does the Callaway and Sant'Anna estimator do instead?

It estimates a separate effect for every cohort (the units first treated in the same period) in every period, each from a clean comparison with never-treated or not-yet-treated units, and only then averages them with weights you can see: overall, by time since treatment (the event study) or by cohort.

What data do I need?

A long, balanced panel: one row per unit and period, a numeric period column, the outcome, and either the first period each unit was treated (0 or blank for never) or a 0/1 treatment column that switches on and stays on. Units missing a period are dropped. Several periods before treatment make the pre-trends check meaningful.

How do I check parallel trends?

Look at the pre-treatment points of the event study: if the treated cohorts were already drifting away from the comparison group before treatment, the estimates left of zero will not hover around zero. The joint Wald pre-test summarises them. Passing it does not prove parallel trends, and a small sample can pass it by having little power.

Why does the tool warn about one-unit cohorts?

With one unit in a cohort there is no way to estimate how much that unit moves around by chance, so the standard errors of its estimates reflect only the comparison group and are too small. The pre-test is not reported in that case. Setting the minimum cohort size to 2 leaves such cohorts out.

Is my data uploaded anywhere?

No. The file is read and analysed by JavaScript in this browser tab. There are no accounts, cookies or analytics vendors.