What does a state estimator tuned in a twin with the wrong noise levels cost in the real world?

research/filter-twin

An estimator tuned in a twin with wrong noise levels: exact real MSE of the twin's Kalman gain, optimistic claim when the twin is too quiet (+121% at 10×) vs pessimistic when too noisy, no instability cliff, and a Mehra innovation retune with regret ∝ 1/n.

10 tests

What it found

For a scalar Kalman filter (plant x′=ax+w, sensor y=x+v) the real error under the twin’s gain K is exactly ((1−K)²Q+K²R)/(1−(1−K)²a²). With a=0.9 and Q=R=1, a twin 10× too quiet claims MSE 0.215 but the real MSE is 1.32 (+121%), while one 10× too noisy is slightly pessimistic (+42%); at 100× the quiet twin costs 5.0 against 0.64, and for a random walk quiet-twin regret grows as m^−1/2. Regret is locally c(ln m)². Negative result: for an unstable plant the twin’s gain never destabilises the filter. The lag-1 innovation covariance is zero iff the gain is optimal, and a one-shot moment retune from real innovations cut regret from 2.15 to 0.026 at n=200 and 0.0002 at n=25,000 (simulation, ∼1/n). Stylised: scalar linear-Gaussian system, stationary, known dynamics.

A figure from its own output

Figure produced by research/filter-twin: mis_set_ratio — filter-twin/experiments/mis_set_ratio.svg, written by the project's own experiment code.

Paper, code and results

More in Digital twins & sim-to-real

What does a controller designed in a twin with the wrong actuator gain cost in the real world?A controller designed in a twin with the wrong actuator gain: exact real LQR cost, an instability band the twin does not warn about, closed-loop non-identifiability without dither, and a dithered retune with matched regret law.
How long does a simulator without actuator limits say an unstable plant stays in control?A limit-free twin of an unstable plant with a saturating actuator overstates mean time to loss of control by 6,295× at noise sd 1 (1.5×10⁶ vs 245 steps), an exponential-in-1/s² gap (slope ≈3.7 real vs ≈12.7–13.2 twin), though both rank the deadbeat gain best.
What does a Kalman filter tuned in an ideal-sensor twin cost on a quantizing real sensor?A Kalman filter tuned in an ideal-sensor twin, run on a quantizing sensor: twin gain costs +11% (R=1, D=4) to +64% (R=0.01), Sheppard R+D²/12 retune is within 1% of best, but its MSE claim under-predicts by up to 43% once the step exceeds the state's scale.
How should a simulation budget be split across twin fidelities?Splitting a simulation budget across twin fidelities (multilevel Monte Carlo): smooth quantity saves 3.6×–897× (rates α=1, β=2), a failure probability only β=1 so multilevel loses at loose targets (0.4×) and reaches 7.7× at 1e-4; equal samples per level cost 156×/3.9× the optimum.
How many real steps does a twin-pretrained policy save, and which twin errors does fine-tuning inherit?Fine-tuning a twin-pretrained policy on the real plant: exact SGD risk, a twin start saves 64–67% of real steps at 0.6× the cold error but is negative transfer beyond 1×, and twin error decays only as b²/(4eηk) (flat directions inherited: 13 vs 1,541 steps at equal initial loss).
How far ahead should a controller planned in a twin with the wrong plant pole look?How far ahead a controller planned in a wrong-pole twin should look: exact real cost per horizon is non-monotone when the twin overstates the pole (1.81 at H=2 vs 3.97 at H=60, optimum 1.48), the twin's own claim ranks horizons backwards, exact two-step stability edge, and a Riccati terminal cost removes the effect.

All 164 Digital twins & sim-to-real projects →

Other research topics

Proper scoring & elicitation28 Effort & peer prediction9 Verification games & slashing19 Detection & tolerances14 Markets as routers13 Prediction-market design9 Paying contributors8 Decentralised training systems23 Digital twins & sim-to-real164 LLM chip design loops40 Robot manufacturing loops53 Ambient AI assistant46 Self-replicating manufacturing34 Simulation-first design64 Engineered clothing45 Task-specific AI accelerators40 Algorithmic chip-design loops42 Open-source purpose-built robots44 The software-engineering loop43 Adversarial human imitation41 Open-source residential construction41 Autonomous multi-angle capture34 Sensing, imaging and 3D31 Energy supply and dense energy storage39 Robot societies among people31 Agentic co-design of low-cost robots27 Laundry-hanging robots25 Machine learning on qualitative data23 Robots on the building site27 Civilizations & group psychology1 History & human sciences1

Produced with AI assistance (Claude). Results are from the code in this repository and are simulated or stylised unless the abstract says otherwise; nothing here implies affiliation with or endorsement by any lab, researcher or company cited. MIT licensed.