How much of an old task must a continual learner store, and is it better to replay a little often or a lot rarely?

What it found

If only m of an old task’s r examples are stored, replaying them with a fresh task is “project out the stored subspace, then take a plain step in dimension d−m”. That gives an exact 2×2 mean-state matrix interpolating no replay (m=0) and full replay (m=r, matched to replay-law to 12 digits), agreeing with simulation at every step. One replay at its best step recovers a fraction of the full saving that is nearly linear in m/r (0.23, 0.47, 0.72 at ¼, ½, ¾ for d=12, r=4; within 0.004 of m/r at d=128, r=8). At a fixed rate, half the memory buys 57–65% of the saving. At a fixed budget of examples per step, one large replay beats many small ones, but only by 0.1–2 points of total forgetting in wide models and up to 6 in a tight one (d=12, r=4), so buffer size matters more than schedule. Stylised: realizable linear regression, exact convergence, one tracked task.

A figure from its own output

Paper, code and results

• README and method
• White paper
• Experiments and results
• Code
• Tests

More in Decentralised training systems

• When continual-learning tasks conflict, how much does a task-aware router of modules actually buy?Task-aware routing when continual-learning tasks conflict: the long-run floor is (2r/d)[τ_w²+τ_b²(1−Purity)] with purity = 1 − the router's Gini impurity, random modules buy nothing, a perfect router removes (m−1)/(K−1) of the between-cluster floor, and a 10% routing error gives back 36%.
• How should the outer optimiser and sync interval be set when workers run H local steps per round?DiLoCo-style outer momentum: H inner steps precondition the outer quadratic (κ_H ≈ κ/H), exact per-mode rates, optimal momentum falls 0.88 → 0.01 with H (fixed 0.7/0.9 Nesterov needs 4–8× more rounds), and the wallclock-optimal sync interval is H* ≈ C (sync cost in inner steps), worth ≈ √C/2.
• How much gradient noise survives in DiLoCo-style local SGD, and how should the outer step, momentum, worker count and sync interval be set against it?Noise floor of DiLoCo-style local SGD: exact stationary loss αV(1+β)/((1−β)s(2(1+β)−αs)), independent of H at α=1, momentum free at equal effective step, near-linear speedup in M, and a noise-limited sync interval that grows as the floor target falls.
• What happens to local SGD when workers drop in and out?Local SGD when workers drop in and out: exact floors for participant averaging (E[1/K|K≥1]) and expected-count normalisation (stable iff αsc<2), at matched speed the fixed normaliser is quieter by E[K]E[1/K] at small steps, and reliable workers bias the fixed point unless inverse-propensity weighted.
• What does compressing the pseudo-gradient cost local SGD, and is it better than syncing less often?Local SGD with compressed displacements: exact floor (1+ω) and stability limit αs(1+ω/N)<2 for rand-k (more workers restore the step, not the floor), closed form for norm-scaled quantisation, and at equal bytes syncing less often beats compressing.
• How much outer step does overlapping communication with compute cost, and how many rounds should be in flight?Overlapping communication in local SGD with τ-round-late outer updates: exact stable step 2 sin(π/(4τ+2)) (shrinks ~1/τ), double-root rate τ/(τ+1), exact noise floor matched to simulation, free at small steps, wall-clock-optimal delay ⌈L/T_c⌉ (1.9–7.5× faster than no overlap).

All 23 Decentralised training systems projects →

Other research topics

Proper scoring & elicitation28Effort & peer prediction9Verification games & slashing19Detection & tolerances14Markets as routers13Prediction-market design9Paying contributors8Decentralised training systems23Digital twins & sim-to-real164LLM chip design loops40Robot manufacturing loops53Ambient AI assistant46Self-replicating manufacturing34Simulation-first design64Engineered clothing45Task-specific AI accelerators40Algorithmic chip-design loops42Open-source purpose-built robots44The software-engineering loop43Adversarial human imitation41Open-source residential construction41Autonomous multi-angle capture34Sensing, imaging and 3D31Energy supply and dense energy storage39Robot societies among people31Agentic co-design of low-cost robots27Laundry-hanging robots25Machine learning on qualitative data23Robots on the building site27Civilizations & group psychology1History & human sciences1

Produced with AI assistance (Claude). Results are from the code in this repository and are simulated or stylised unless the abstract says otherwise; nothing here implies affiliation with or endorsement by any lab, researcher or company cited. MIT licensed.