How many designs should a digital twin shortlist for real-plant verification?

research/twin-screen

Shortlisting K designs in a twin and verifying the top m on the real plant under a fixed trial budget: twin-only value exactly ρc_K, best m falls from 8 to 2 as ρ rises 0.6→0.95 (K=50, B=20), and a useless twin still gains 0.12 from budget concentration alone. Stylised.

6 tests

What it found

A twin ranks K=50 designs and a fixed budget of 20 noisy real trials verifies only the top m. The twin alone is worth exactly ρc_K (c_K=2.249), and the best shortlist shrinks as the twin improves (m=8 at ρ=0.6, 2 at 0.95), reaching 1.87 and 2.17 sd against 1.20 for real-only testing and an oracle 2.25. A useless twin still gains 0.12 by concentrating the budget on fewer designs, so that part is not the twin's. Stylised Gaussian model, even budget split, Monte Carlo.

A figure from its own output

Figure produced by research/twin-screen: value_vs_shortlist — twin-screen/experiments/value_vs_shortlist.svg, written by the project's own experiment code.

Paper, code and results

More in Digital twins & sim-to-real

How far should a real-plant estimate be pulled toward its digital twin?Pulling a real-plant parameter estimate toward its twin (James–Stein): exact risk d−(d−2)²E[1/(d−2+2K)], never worse in total for d≥3, perfect twin risk 2 at any d, worth n+σ²/τ² real samples at large d, but one mismatched parameter takes ≈d/4 noise variances (25.5 at d=100) unless shrinkage is blockwise (1.65 at blocks of 5, d=30).
What does a twin that assumes the radar is aligned with the body get wrong about dead reckoning, and can loop-closure validation see it?A radar twin assuming the sensor is body-aligned: planar dead reckoning is exactly the true path rotated by the mount yaw (error 2 sin(e/2)·|displacement|, bounded, zero after loops, so loop-closure validation is blind), closed-form rotation fit with std σ/√Σ|p|², but a pitched mount drifts vertically laps·2πR sin e.
How many real rollouts does a biased digital twin save when certifying a policy?Certifying a policy with a biased digital twin: exact twin/plant cost correlation, optimal twin-to-real rollout split and speed-up (matched by simulation), bias first order but correlation second order in model error so the replayable-disturbance fraction caps the twin's value, and twin-only intervals cover ~0%.
What does a Gaussian-clutter radar twin promise about detection, not just false alarms?What a Gaussian-clutter radar twin promises about detection: exact Pfa/Pd laws for CA- and OS-CFAR in textured clutter, OS inflated like CA, an SNR shortfall (6.3 dB at ν=2) larger than the threshold repair, and no CA/OS ranking reversal.
How do you estimate a controller's real-world cost with a biased digital twin?Estimating a controller's real cost with a biased digital twin as a control variate: cost correlation in closed form (0.985 → 0.472 as the twin's gain error grows while its bias grows 300×), exactly λ² under partial disturbance replay, optimal twin/real ratio √(c_rρ²/(c_t(1−ρ²))) and break-even twin price, simulated variance ratio 0.228 vs 0.225 predicted.
How fast can a deployed digital twin be caught going stale, and when is it invisible?Anytime-valid monitoring of a deployed twin: e-process on closed-loop residuals (2.3% vs 51.6% false alarms under peeking), rate = twin-transfer score gap, an invisible blind line Δa = kΔb that needs dither v, and when dithering pays.

All 164 Digital twins & sim-to-real projects →

Other research topics

Proper scoring & elicitation28 Effort & peer prediction9 Verification games & slashing19 Detection & tolerances14 Markets as routers13 Prediction-market design9 Paying contributors8 Decentralised training systems23 Digital twins & sim-to-real164 LLM chip design loops40 Robot manufacturing loops53 Ambient AI assistant46 Self-replicating manufacturing34 Simulation-first design64 Engineered clothing45 Task-specific AI accelerators40 Algorithmic chip-design loops42 Open-source purpose-built robots44 The software-engineering loop43 Adversarial human imitation41 Open-source residential construction41 Autonomous multi-angle capture34 Sensing, imaging and 3D31 Energy supply and dense energy storage39 Robot societies among people31 Agentic co-design of low-cost robots27 Laundry-hanging robots25 Machine learning on qualitative data23 Robots on the building site27 Civilizations & group psychology1 History & human sciences1

Produced with AI assistance (Claude). Results are from the code in this repository and are simulated or stylised unless the abstract says otherwise; nothing here implies affiliation with or endorsement by any lab, researcher or company cited. MIT licensed.