How optimistic is the best of K controllers chosen in a noisy digital twin?

research/selection-twin

The best of K controllers picked in a noisy twin: winner's claim optimistic by exactly τ²c_K/√(v²+τ²) (c_K=E max of K normals), under a fixed rollout budget more candidates can make the deployed one worse (best K=5–22), an exact-plant LQ twin's claim falls below what any gain can achieve, and screen-then-verify cuts claim bias 4.5×.

9 tests

What it found

Scoring K candidate controllers in a twin and deploying the argmin is a winner’s-curse selection. With score noise τ around true costs N(m, v²), the winner’s claimed cost is exactly m − √(v²+τ²) c_K and its deployed cost m − v²c_K/√(v²+τ²), so optimism τ²c_K/√(v²+τ²) grows like √(2 ln K) (simulation agrees to within 0.03). With a fixed rollout budget the winner’s deployed cost is non-monotone in K (best K = 5, 8, 22 in three regimes; 5,000 candidates gain 5–10× less). In an exact-plant LQ twin (50 gains, 10 rollouts each) the claim is 8% below the deployed cost and below what any gain can achieve, though the Gaussian formula understates the size (optimism 0.0036 vs 0.0017 predicted). Fresh re-scoring fixes the claim but not the choice, empirical-Bayes shrinkage barely helps, and screen-then-verify cuts claim bias 4.5× (Gaussian model: regret 0.73 → 0.34 at K=100). Stylised: i.i.d. Gaussian score noise, one scalar plant, twin equal to the real plant.

A figure from its own output

Figure produced by research/selection-twin: claimed_vs_real — selection-twin/experiments/claimed_vs_real.svg, written by the project's own experiment code.

Paper, code and results

More in Digital twins & sim-to-real

How many real rollouts does a biased digital twin save when evaluating a robot policy?Evaluating a robot policy with a biased twin as a control variate: unbiased whatever the twin's bias (+0.700 twin-only vs +0.001), exact variance σ²((1−ρ²)/n+ρ²/N), ρ=η for a faithful twin but √η for a noise-free one (1.33× vs 2.0× real-sample gain), worth-it threshold c_T/c_R<(1−√(1−ρ²))²/ρ² (0.072 at ρ=0.5), and 86–125 vs 164 real episodes per policy to rank correctly when twin bias favours the wrong one.
How big a pilot decides whether a biased digital twin is worth using?How big a pilot decides whether a biased twin is worth using: threshold ρ*=2√r/(1+r) with a gain linear in ρ−ρ*, so wrong calls are frequent but cheap (regret 0.004–0.016 at ρ=0.5), decision-regret law f·slope·(1−ρ*²)²/(2m) (ratio 0.63→1.00 for m=10→160), plug-in allocation ≈0.27–0.30/m, and an adaptive two-stage design that stays unbiased (|bias| ≤ 0.002) with 0.946–0.955 coverage.
Does giving two policies the same seed in a digital twin actually pair them?Seed-paired (common random numbers) policy comparison in a twin: correlation is exactly lost to random-stream desynchronisation (flat score 0.95→0.77, terminal score 0.36→0.001 at q=0.05→0.3), a sizing trap (power 0.11–0.85 vs nominal 0.90), and counter-keyed draws restore ρ=1.
What overshoot does a controller tuned in a fixed-step digital twin really have?What overshoot a controller tuned in a fixed-step twin really has: exact per-step decay laws for explicit/semi-implicit/implicit Euler, damping bias flips sign at ζ=1/√2 (explicit, implicit), a 10% twin target delivers 0.105–0.192 real overshoot (implicit/semi-implicit) or 0.036–0.091 (explicit), error first order in h, Richardson repair.
What does a too-narrow scenario generator do to a digital twin's estimate of a rare failure?A too-narrow scenario generator in a twin makes rare-failure estimates silent: reweighted variance finite iff s²>½, 0.6× width claims 2e-7 for a real 1.2e-3 and reports zero failures in all runs, optimal width ≈1.05·a needs 38× fewer runs than plain Monte Carlo, and a defensive real-prior mixture is finite but no better.
What does a state estimator tuned in a twin with the wrong noise levels cost in the real world?An estimator tuned in a twin with wrong noise levels: exact real MSE of the twin's Kalman gain, optimistic claim when the twin is too quiet (+121% at 10×) vs pessimistic when too noisy, no instability cliff, and a Mehra innovation retune with regret ∝ 1/n.

All 164 Digital twins & sim-to-real projects →

Other research topics

Proper scoring & elicitation28 Effort & peer prediction9 Verification games & slashing19 Detection & tolerances14 Markets as routers13 Prediction-market design9 Paying contributors8 Decentralised training systems23 Digital twins & sim-to-real164 LLM chip design loops40 Robot manufacturing loops53 Ambient AI assistant46 Self-replicating manufacturing34 Simulation-first design64 Engineered clothing45 Task-specific AI accelerators40 Algorithmic chip-design loops42 Open-source purpose-built robots44 The software-engineering loop43 Adversarial human imitation41 Open-source residential construction41 Autonomous multi-angle capture34 Sensing, imaging and 3D31 Energy supply and dense energy storage39 Robot societies among people31 Agentic co-design of low-cost robots27 Laundry-hanging robots25 Machine learning on qualitative data23 Robots on the building site27 Civilizations & group psychology1 History & human sciences1

Produced with AI assistance (Claude). Results are from the code in this repository and are simulated or stylised unless the abstract says otherwise; nothing here implies affiliation with or endorsement by any lab, researcher or company cited. MIT licensed.