How big a pilot decides whether a biased digital twin is worth using?

research/twin-pilot

How big a pilot decides whether a biased twin is worth using: threshold ρ*=2√r/(1+r) with a gain linear in ρ−ρ*, so wrong calls are frequent but cheap (regret 0.004–0.016 at ρ=0.5), decision-regret law f·slope·(1−ρ*²)²/(2m) (ratio 0.63→1.00 for m=10→160), plug-in allocation ≈0.27–0.30/m, and an adaptive two-stage design that stays unbiased (|bias| ≤ 0.002) with 0.946–0.955 coverage.

6 tests

What it found

The twin pays iff its correlation with the real return exceeds ρ*=2√r/(1+r) (0.426 at cost ratio r=0.05), and the gain is linear in ρ−ρ* there, so a small pilot's wrong calls are frequent but cheap (probability 0.36–0.11 at ρ=0.5 for m=10–160 pairs, regret 0.004–0.016 of real-only variance). Decision regret follows f·slope·(1−ρ*²)²/(2m) (simulated/law 0.63 at m=10, 1.00 at 160) and plug-in allocation costs ≈0.27–0.30/m extra variance. The adaptive two-stage design (pilot reused) stays unbiased (|bias| ≤ 0.002 against twin bias 0.7) with 95% intervals covering 0.946–0.955. Stylised: Gaussian returns, exact context replay, asymptotic law, no robot data.

A figure from its own output

Figure produced by research/twin-pilot: pwrong — twin-pilot/experiments/pwrong.svg, written by the project's own experiment code.

Paper, code and results

More in Digital twins & sim-to-real

Does giving two policies the same seed in a digital twin actually pair them?Seed-paired (common random numbers) policy comparison in a twin: correlation is exactly lost to random-stream desynchronisation (flat score 0.95→0.77, terminal score 0.36→0.001 at q=0.05→0.3), a sizing trap (power 0.11–0.85 vs nominal 0.90), and counter-keyed draws restore ρ=1.
What overshoot does a controller tuned in a fixed-step digital twin really have?What overshoot a controller tuned in a fixed-step twin really has: exact per-step decay laws for explicit/semi-implicit/implicit Euler, damping bias flips sign at ζ=1/√2 (explicit, implicit), a 10% twin target delivers 0.105–0.192 real overshoot (implicit/semi-implicit) or 0.036–0.091 (explicit), error first order in h, Richardson repair.
What does a too-narrow scenario generator do to a digital twin's estimate of a rare failure?A too-narrow scenario generator in a twin makes rare-failure estimates silent: reweighted variance finite iff s²>½, 0.6× width claims 2e-7 for a real 1.2e-3 and reports zero failures in all runs, optimal width ≈1.05·a needs 38× fewer runs than plain Monte Carlo, and a defensive real-prior mixture is finite but no better.
What does a state estimator tuned in a twin with the wrong noise levels cost in the real world?An estimator tuned in a twin with wrong noise levels: exact real MSE of the twin's Kalman gain, optimistic claim when the twin is too quiet (+121% at 10×) vs pessimistic when too noisy, no instability cliff, and a Mehra innovation retune with regret ∝ 1/n.
What does a controller designed in a twin with the wrong actuator gain cost in the real world?A controller designed in a twin with the wrong actuator gain: exact real LQR cost, an instability band the twin does not warn about, closed-loop non-identifiability without dither, and a dithered retune with matched regret law.
How long does a simulator without actuator limits say an unstable plant stays in control?A limit-free twin of an unstable plant with a saturating actuator overstates mean time to loss of control by 6,295× at noise sd 1 (1.5×10⁶ vs 245 steps), an exponential-in-1/s² gap (slope ≈3.7 real vs ≈12.7–13.2 twin), though both rank the deadbeat gain best.

All 164 Digital twins & sim-to-real projects →

Other research topics

Proper scoring & elicitation28 Effort & peer prediction9 Verification games & slashing19 Detection & tolerances14 Markets as routers13 Prediction-market design9 Paying contributors8 Decentralised training systems23 Digital twins & sim-to-real164 LLM chip design loops40 Robot manufacturing loops53 Ambient AI assistant46 Self-replicating manufacturing34 Simulation-first design64 Engineered clothing45 Task-specific AI accelerators40 Algorithmic chip-design loops42 Open-source purpose-built robots44 The software-engineering loop43 Adversarial human imitation41 Open-source residential construction41 Autonomous multi-angle capture34 Sensing, imaging and 3D31 Energy supply and dense energy storage39 Robot societies among people31 Agentic co-design of low-cost robots27 Laundry-hanging robots25 Machine learning on qualitative data23 Robots on the building site27 Civilizations & group psychology1 History & human sciences1

Produced with AI assistance (Claude). Results are from the code in this repository and are simulated or stylised unless the abstract says otherwise; nothing here implies affiliation with or endorsement by any lab, researcher or company cited. MIT licensed.