How much is one long digital-twin run worth when its output is serially correlated?

What it found

Twin output such as queue waits is autocorrelated, but a run is usually summarised as mean ± 1.96 s/√n. For AR(1) output the run-mean variance is exact (matched to simulation), so 1000 draws at φ=0.9 are worth 53. For an M/M/1 lab instrument the variance of the run mean is 9.7×, 33× and 363× the iid value at ρ=0.5, 0.7, 0.9: from 10⁴ jobs the naive 95% interval covers the true mean wait 53%, 26% and 9% of the time, and a twin “certifies” an SLA of 8 against a true mean of 9 in 26% of runs. Higher load needs a longer run (470 jobs by the iid formula, 170,268 exactly at ρ=0.9). Batch means restore coverage (94–95%) only when batches outlast the correlation time; it is 53.5% at φ=0.99 and 82.5% at ρ=0.9, n=10⁴. Stylised (Gaussian AR(1) and M/M/1, simulated “real” system).

A figure from its own output

Paper, code and results

• README and method
• White paper
• Experiments and results
• Code
• Tests

More in Digital twins & sim-to-real

• How much of a digital-twin run should be thrown away because it started in the wrong state?A twin started in the wrong state: exact AR(1) warm-up bias and MSE-optimal deletion (≈ one relaxation time, ≤1.5× gain), M/M/1 empty-start bias (−21% at ρ=0.9, n=500; 1% at 10⁴), and a negative result: MSER-5 truncation leaves more bias and more false SLA certifications than fixed deletion.
• How wrong is a twin's localisation error when real ranges have non-line-of-sight bias?A twin with Gaussian range noise vs real non-line-of-sight bias: exact rms law GDOP·√(σ²+p(2−p)m²) (4.5–7.3× the twin's error, zero mean), a predictable bias for a blocked side, anchor counts 4 → 28–46 for a 1 m / 95% corridor, and a trimming fix that costs 24% when NLOS is absent.
• Can a twin identified from logs of the deployed controller tell you whether a different controller is stable?A twin identified from logs of the deployed controller: held-out error is 1.00σ² at every dither level while the predicted pole under a new gain has sd (σ/√n)√((k−k')²/τ²+1/v) (0.037→0.82), no dither gives a confident wrong twin (b̂=−0.22 vs +1), point-rule false stability certification 3–48% vs 0.1–4.7% for the OLS bound.
• How much of a digital twin's error is that it was fitted to finite real data, however long it is simulated?A twin fitted to finite real data: exact delta-method error √(1+(2−ρ)²)/((1−ρ)√n) of its steady-state wait, data needed grows as (1−ρ)⁻² (22 100 per stream for 10% at ρ=0.9), exact instability probability, false SLA certification 36–46% fixed by bootstrap/posterior bounds (delta bound under-covers).
• For a fixed budget, how much should a fitted digital twin spend on real data and how much on simulation length?Real data or more simulation for a fitted twin: relative variance a(ρ)/n + b(ρ)/m (matches simulation), one real pair ≈ 2 simulated jobs at any load, optimal real-data share 0.57–0.97 by cost ratio, and a simulation-only interval whose real coverage falls 0.79→0.17 as the run lengthens while the input-aware one stays 0.92–0.95.
• When should a twin run that counts rare failures stop?When to stop a twin run counting rare failures: sequential Wald stopping halts on zero failures (90% of runs at p=10⁻³, coverage 9.6%), inverse sampling to m failures gives exactly unbiased (m−1)/(N−1) with relative sd ≈1/√(m−2) and an exact interval (95.0–95.5%), while a fixed run planned for the wrong p delivers 63–200% half-width.

All 164 Digital twins & sim-to-real projects →

Other research topics

Proper scoring & elicitation28Effort & peer prediction9Verification games & slashing19Detection & tolerances14Markets as routers13Prediction-market design9Paying contributors8Decentralised training systems23Digital twins & sim-to-real164LLM chip design loops40Robot manufacturing loops53Ambient AI assistant46Self-replicating manufacturing34Simulation-first design64Engineered clothing45Task-specific AI accelerators40Algorithmic chip-design loops42Open-source purpose-built robots44The software-engineering loop43Adversarial human imitation41Open-source residential construction41Autonomous multi-angle capture34Sensing, imaging and 3D31Energy supply and dense energy storage39Robot societies among people31Agentic co-design of low-cost robots27Laundry-hanging robots25Machine learning on qualitative data23Robots on the building site27Civilizations & group psychology1History & human sciences1

Produced with AI assistance (Claude). Results are from the code in this repository and are simulated or stylised unless the abstract says otherwise; nothing here implies affiliation with or endorsement by any lab, researcher or company cited. MIT licensed.