Which robust paired test still catches a wrong digital twin when noise has infinite variance or whole runs go bad?

What it found

Follow-up to free-robustpaired on the same stylised pendulum (10% Coulomb-friction error, n = 50–200 matched trajectories, 400 null replicates). With Student-t(2) cells (infinite variance) the mean squared residual has power about 0.05 at every n, while median and trimmed squared residuals reach 1.00 and a Huber loss with a MAD-estimated scale reaches 0.70–1.00 (0.20–0.53 with the scale fixed at 0.02). When 10% of runs are 10× noisier, every cell-level statistic stalls (0.04–0.40) while the median over runs of per-run mean squares reaches 0.89–1.00; that statistic is nearly the worst under t(2) cells (0.11–0.19). No statistic wins both, and under Gaussian noise the median pays at n = 50 (0.72 against 0.98). Stylised.

A figure from its own output

Paper, code and results

• README and method
• Experiments and results
• Tests

More in Digital twins & sim-to-real

• Which paired test of a digital twin survives both heavy-tailed cells and whole bad runs?Which paired test of a digital twin survives both heavy-tailed cells and whole bad runs? Stylised pendulum, 10% Coulomb error: a two-level statistic (trimmed mean of squared residuals within each run, then trimmed mean across runs) has power 0.99–1.00 at n = 50 under t(2) cells, bad runs at 0.86 and both at once, where the earlier single-level statistics each fail one case. Costs power under Gaussian noise (0.84 against 0.91).
• Which significance rule gives a classifier two-sample test the right false-rejection rate on a digital twin?Which significance rule gives a classifier two-sample test the right false-rejection rate on a digital twin? Stylised pendulum: cross-validated rules falsely reject 0.06 to 0.13 at a nominal 0.05, a holdout rank-sum test holds 0.04 to 0.06 at roughly twice the data cost. Preliminary, simulated only.
• Does a permutation test on the cross-validated statistic keep its size and beat the holdout rule's power on a digital twin?Does a permutation test on the cross-validated statistic keep its size and beat the holdout rule's power on a digital twin? Stylised pendulum: null rates 0.040 and 0.020, power 0.367 vs 0.183 for holdout at n=200 (Coulomb -10%).
• What does a quadratic thrust curve do to a controller designed in a linear-thrust twin?A controller that inverts a linear-thrust twin meets a quadratic thrust curve: realised acceleration is a+a²/(4g), so climbs overshoot more (0.046→0.095) and descents less, and steps below p≈0.5 hardly notice.
• Does inverting a quadratic thrust curve exactly fix the error a linear-thrust twin makes, and what if the thrust exponent is also wrong?Exact square-root thrust inverse vs the linear one on a quadratic-thrust vehicle: up/down overshoot gap 3.00→0.09 points with the right model, but only 3.67→1.65 when the thrust exponent is misidentified (2.5 vs 2).
• How much does a wrong process-noise guess cost a Kalman filter?How much does a wrong process-noise guess cost a Kalman filter? Exact scalar steady state: under-guessing q by ×100 costs ×7.4 MSE, over-guessing by ×100 only ×1.6 (bounded by r), and the under-guessing filter reports 5× too-small error bars. Stylised.

All 164 Digital twins & sim-to-real projects →

Other research topics

Proper scoring & elicitation28Effort & peer prediction9Verification games & slashing19Detection & tolerances14Markets as routers13Prediction-market design9Paying contributors8Decentralised training systems23Digital twins & sim-to-real164LLM chip design loops40Robot manufacturing loops53Ambient AI assistant46Self-replicating manufacturing34Simulation-first design64Engineered clothing45Task-specific AI accelerators40Algorithmic chip-design loops42Open-source purpose-built robots44The software-engineering loop43Adversarial human imitation41Open-source residential construction41Autonomous multi-angle capture34Sensing, imaging and 3D31Energy supply and dense energy storage39Robot societies among people31Agentic co-design of low-cost robots27Laundry-hanging robots25Machine learning on qualitative data23Robots on the building site27Civilizations & group psychology1History & human sciences1

Produced with AI assistance (Claude). Results are from the code in this repository and are simulated or stylised unless the abstract says otherwise; nothing here implies affiliation with or endorsement by any lab, researcher or company cited. MIT licensed.