How many real rollouts does a biased digital twin save when certifying a policy?

research/twin-control-variate

Certifying a policy with a biased digital twin: exact twin/plant cost correlation, optimal twin-to-real rollout split and speed-up (matched by simulation), bias first order but correlation second order in model error so the replayable-disturbance fraction caps the twin's value, and twin-only intervals cover ~0%.

9 tests

What it found

A twin evaluated alone is biased however many rollouts you run (nominal 95% intervals cover the truth 0–0.4% of the time at a 12% bias, 20% gain error); used as a control variate it removes variance without bias. In scalar LQ the rollout cost is a Gaussian quadratic form, so the twin/plant correlation ρ = λ tr(GG′)/√(tr G² tr G′²) is exact, the optimal twin-to-real ratio is ρ/√((1−ρ²)w) and the speed-up 1/(√(1−ρ²)+ρ√w)², matched by simulation to a few percent. Twin gain error is first order in the bias (5% error → 2.7%) but second order in 1−ρ, so the twin’s value is capped by the fraction λ of the real disturbance it can replay: a 5% certificate costs 168 real-only, 28 with λ = 0.95 and 163 with λ = 0.3. Twin-only beats real-only only below σ²/bias² real rollouts (35 at 10% error). Stylised.

A figure from its own output

Figure produced by research/twin-control-variate: speedup — twin-control-variate/experiments/speedup.svg, written by the project's own experiment code.

Paper, code and results

More in Digital twins & sim-to-real

What does a Gaussian-clutter radar twin promise about detection, not just false alarms?What a Gaussian-clutter radar twin promises about detection: exact Pfa/Pd laws for CA- and OS-CFAR in textured clutter, OS inflated like CA, an SNR shortfall (6.3 dB at ν=2) larger than the threshold repair, and no CA/OS ranking reversal.
How do you estimate a controller's real-world cost with a biased digital twin?Estimating a controller's real cost with a biased digital twin as a control variate: cost correlation in closed form (0.985 → 0.472 as the twin's gain error grows while its bias grows 300×), exactly λ² under partial disturbance replay, optimal twin/real ratio √(c_rρ²/(c_t(1−ρ²))) and break-even twin price, simulated variance ratio 0.228 vs 0.225 predicted.
How fast can a deployed digital twin be caught going stale, and when is it invisible?Anytime-valid monitoring of a deployed twin: e-process on closed-loop residuals (2.3% vs 51.6% false alarms under peeking), rate = twin-transfer score gap, an invisible blind line Δa = kΔb that needs dither v, and when dithering pays.
How far can DiLoCo reports be compressed?Compressing DiLoCo reports: unbiased compression multiplies the contraction (c = 1+ω/M, stable iff αsc<2, participation is ω = 1/p−1), dithered quantisation only adds Δ²/12 and keeps stability, and at fixed bandwidth quantisation has an interior optimum near 1.3 bits per coordinate (~4× below 8-bit reports) while sparsification favours few full reports.
What does it cost to send only a fraction of the outer update, and does error feedback help?Sending a fraction p of the outer update: unbiased rescaling costs 1/p in noise floor, error feedback costs only 1+αs(1/p−1) at small steps but caps contraction at √(1−p); exact closed form, stability limits, horizon losses.
Can a DiLoCo-style run hide its synchronisation by applying the outer update rounds late?Overlapping the DiLoCo sync by applying the outer step τ rounds late does not pay: stable iff αs < 2 sin(π/(4τ+2)), best rate 1−2 sin(π/(4τ+2))/κ_H (exactly κ_H/(κ_H+1) at τ=1), momentum is useless once τ≥1 (69 → 1388 rounds at κ_H=100), hidden-sync gain ≤ (τ+1) sin(π/(4τ+2)) ≤ 1 (search: 1.5–4.9× slower than blocking averaging), fixed if the fresh pseudo-gradient has weight >½.

All 164 Digital twins & sim-to-real projects →

Other research topics

Proper scoring & elicitation28 Effort & peer prediction9 Verification games & slashing19 Detection & tolerances14 Markets as routers13 Prediction-market design9 Paying contributors8 Decentralised training systems23 Digital twins & sim-to-real164 LLM chip design loops40 Robot manufacturing loops53 Ambient AI assistant46 Self-replicating manufacturing34 Simulation-first design64 Engineered clothing45 Task-specific AI accelerators40 Algorithmic chip-design loops42 Open-source purpose-built robots44 The software-engineering loop43 Adversarial human imitation41 Open-source residential construction41 Autonomous multi-angle capture34 Sensing, imaging and 3D31 Energy supply and dense energy storage39 Robot societies among people31 Agentic co-design of low-cost robots27 Laundry-hanging robots25 Machine learning on qualitative data23 Robots on the building site27 Civilizations & group psychology1 History & human sciences1

Produced with AI assistance (Claude). Results are from the code in this repository and are simulated or stylised unless the abstract says otherwise; nothing here implies affiliation with or endorsement by any lab, researcher or company cited. MIT licensed.