How wrong is the thinnest obstacle a lidar twin certifies as detectable when real beam dropout is clustered?

research/dropout-twin

A lidar twin with i.i.d. beam dropout certifies the thinnest detectable obstacle: real clustered dropout (same rate, lag-1 correlation 0.6) misses 41× more at the certified width and the range claim is 4× too long (exact Markov miss law, p·q^(m−1)); a fitted persistence recovers the median width but needs a margin.

7 tests

What it found

A thin obstacle covers m adjacent lidar beams and is detected if at least k return. The twin drops beams independently at 10%; the real sensor drops them in bursts (a Markov chain with the same rate and lag-1 correlation λ). The exact miss law is a small DP, and for k = 1 the real/twin ratio is (1 + λ(1−p)/p)^m−1. At the width certified for miss 10⁻³ (m = 3) the real miss is 0.041 (λ = 0.6) to 0.067 (λ = 0.8), and the claimed detection range is 4× to 8.3× too long. Inflating the drop rate to match one width is wrong at the others. Fitting the chain from a flat-target log recovers the median width but 40% of repeats still miss the target at 5,000 beams; two extra beams cut that to 1%. Stylised: one scan, a simulated ‘real’ sensor, no lidar data.

A figure from its own output

Figure produced by research/dropout-twin: range_ratio — dropout-twin/experiments/range_ratio.svg, written by the project's own experiment code.

Paper, code and results

More in Digital twins & sim-to-real

Can a digital twin shorten a real-world safety certification?Can a twin shorten a real-world safety certification? Worst-case size of any k ≤ c rule is exactly P(Bin(n,ε) ≤ c), so a twin prior cannot cut real trials at a fixed level (299 exact zero-failure trials; twin prior reaches 133 only at size 0.26); wrong twins inflate size or block certification.
How optimistic is the cost a planner quotes from its own twin rollouts?The cost a planner quotes from its own twin rollouts is optimistic by exactly e_K·s (winner's curse; 95% intervals cover 60% at K=20), fresh rollouts fix the sampling part at a selection-quality price, but not the twin's own model error −e_K τ²/√(τ²+s²).
How many retries does a grasp need when the simulator treats attempts as independent?How many retries a grasp needs when the twin treats attempts as independent: exact Beta-mixture failure law (power law k^{−a} vs geometric), twin budget 6 claims 0.004 and delivers 0.010–0.151, real budget 7–571, closed-form give-up rule, and a two-attempts-per-object repair that still needs a margin.
How optimistic is the best of K controllers chosen in a noisy digital twin?The best of K controllers picked in a noisy twin: winner's claim optimistic by exactly τ²c_K/√(v²+τ²) (c_K=E max of K normals), under a fixed rollout budget more candidates can make the deployed one worse (best K=5–22), an exact-plant LQ twin's claim falls below what any gain can achieve, and screen-then-verify cuts claim bias 4.5×.
How many real rollouts does a biased digital twin save when evaluating a robot policy?Evaluating a robot policy with a biased twin as a control variate: unbiased whatever the twin's bias (+0.700 twin-only vs +0.001), exact variance σ²((1−ρ²)/n+ρ²/N), ρ=η for a faithful twin but √η for a noise-free one (1.33× vs 2.0× real-sample gain), worth-it threshold c_T/c_R<(1−√(1−ρ²))²/ρ² (0.072 at ρ=0.5), and 86–125 vs 164 real episodes per policy to rank correctly when twin bias favours the wrong one.
How big a pilot decides whether a biased digital twin is worth using?How big a pilot decides whether a biased twin is worth using: threshold ρ*=2√r/(1+r) with a gain linear in ρ−ρ*, so wrong calls are frequent but cheap (regret 0.004–0.016 at ρ=0.5), decision-regret law f·slope·(1−ρ*²)²/(2m) (ratio 0.63→1.00 for m=10→160), plug-in allocation ≈0.27–0.30/m, and an adaptive two-stage design that stays unbiased (|bias| ≤ 0.002) with 0.946–0.955 coverage.

All 164 Digital twins & sim-to-real projects →

Other research topics

Proper scoring & elicitation28 Effort & peer prediction9 Verification games & slashing19 Detection & tolerances14 Markets as routers13 Prediction-market design9 Paying contributors8 Decentralised training systems23 Digital twins & sim-to-real164 LLM chip design loops40 Robot manufacturing loops53 Ambient AI assistant46 Self-replicating manufacturing34 Simulation-first design64 Engineered clothing45 Task-specific AI accelerators40 Algorithmic chip-design loops42 Open-source purpose-built robots44 The software-engineering loop43 Adversarial human imitation41 Open-source residential construction41 Autonomous multi-angle capture34 Sensing, imaging and 3D31 Energy supply and dense energy storage39 Robot societies among people31 Agentic co-design of low-cost robots27 Laundry-hanging robots25 Machine learning on qualitative data23 Robots on the building site27 Civilizations & group psychology1 History & human sciences1

Produced with AI assistance (Claude). Results are from the code in this repository and are simulated or stylised unless the abstract says otherwise; nothing here implies affiliation with or endorsement by any lab, researcher or company cited. MIT licensed.