A stereo twin that adds Gaussian noise to depth gets the median right: what does it get wrong about over-range errors, safety margins and averaging compared with real disparity-space noise?

research/stereo-twin

Gaussian-depth-noise stereo twin vs real disparity-space noise: median exact, gated mean biased by ≈s², over-range errors up to 191× more frequent at 40 m, a twin 3σ margin exceeded 10× as often, mean-depth averaging floors while averaging disparity converges (negative result at the 60 m range floor). Stylised.

14 tests

What it found

Real sensor: disparity noise σ = 0.25 px on a 700 px, 0.12 m rig (matches only above 1 px, so 84 m), depth = reciprocal of disparity; twin: Gaussian depth noise with the linearised sd. The median is exact, but the (gated) mean is biased by about s² (+1.5% at 40 m) and over-estimates of distance are far more common than the twin says: at 40 m P(depth > 1.5Z) is 0.0026 vs 10⁻⁵ (191×), and a twin mean + 3σ margin is exceeded 10× as often as designed. Matching the variance leaves 41× on that tail. Averaging 1024 frames of depth floors at 1.5% error where the twin predicts 0.4%, whereas averaging disparity converges; at 60 m the range floor loses 5.5% of looks and flips or removes these effects (reported as a negative result). Stylised: Gaussian i.i.d. disparity noise, no outliers or calibration error.

A figure from its own output

Figure produced by research/stereo-twin: lost_matches — stereo-twin/experiments/lost_matches.svg, written by the project's own experiment code.

Paper, code and results

More in Digital twins & sim-to-real

A rigid-body twin of a cart carrying liquid predicts no slosh: how wrong is that, what does a slosh-safe move cost, and what does a stale frequency calibration do to it?A rigid twin of a cart carrying liquid predicts zero slosh: minimum-time move leaves 10.7 cm (10× tolerance), slosh-safe move is 55% slower, and a full-tank frequency calibration fails at low fill (28 s vs 8.7 s recalibrated; tuning-free bound 24.7 s).
Can an input shaper make a liquid-carrying cart’s move robust to a stale slosh-frequency calibration, and how fast can it be?Input-shaped moves of a liquid-carrying cart: ZV/ZVD residual sin(πε/2), sin²(πε/2) exactly multiplies the bang-bang residual, so a stale frequency calibration that breaks the null-tuned move (28 s) is harmless to a 5.9 s ZV move; closed-form tolerated error and a fill-band design that spends the tolerance on a shorter shaper.
What is a digital twin worth for training a controller?What a digital twin is worth for training a controller: exact LQ sim-to-real regret with a stability cliff, a validation blind spot (zero score gap yet mistrained control without probing), and a twin worth exactly σ²/(vδ²) real samples at any accuracy.
How wide should domain randomization be, and what does it buy?Domain randomization over a twin, exactly: cliff-limited width, quartic price when the twin was right, cliff pushed out ~15%, but uniform-average training is not minimax and the best width tracks the prior.
What does it cost to keep a digital twin in sync with a drifting plant, and is probing worth it?Keeping a digital twin in sync with a drifting plant: closed-form Kalman tracking variance and control regret (matched to simulation), a frozen twin loses half its edge in ln 2/(2(1−ρ)) steps, probing never pays for one drifting parameter, and a two-parameter closed-loop blind direction gives a mean-field dither law ν* ∝ q^{1/6} that self-exciting certainty-equivalent control mostly makes unnecessary.
How often, and how long, should a digital twin be re-identified as the plant drifts?Re-identifying a drifting twin by dedicated experiments: closed-form interval T*∝q_d^{−2/3} with downtime, noise and drift regret exactly equal at the optimum, excess ∝q_d^{1/3}, a 4× drift misjudgement costs ≤26%, but the second-order optimum destabilises 0.7–15% of cycles and a safe schedule is 37–200× worse than an always-on Kalman twin.

All 164 Digital twins & sim-to-real projects →

Other research topics

Proper scoring & elicitation28 Effort & peer prediction9 Verification games & slashing19 Detection & tolerances14 Markets as routers13 Prediction-market design9 Paying contributors8 Decentralised training systems23 Digital twins & sim-to-real164 LLM chip design loops40 Robot manufacturing loops53 Ambient AI assistant46 Self-replicating manufacturing34 Simulation-first design64 Engineered clothing45 Task-specific AI accelerators40 Algorithmic chip-design loops42 Open-source purpose-built robots44 The software-engineering loop43 Adversarial human imitation41 Open-source residential construction41 Autonomous multi-angle capture34 Sensing, imaging and 3D31 Energy supply and dense energy storage39 Robot societies among people31 Agentic co-design of low-cost robots27 Laundry-hanging robots25 Machine learning on qualitative data23 Robots on the building site27 Civilizations & group psychology1 History & human sciences1

Produced with AI assistance (Claude). Results are from the code in this repository and are simulated or stylised unless the abstract says otherwise; nothing here implies affiliation with or endorsement by any lab, researcher or company cited. MIT licensed.