Does a kernel MMD test catch the same wrong MuJoCo digital twins as a classifier two-sample test?
research/mimic-mmdmujoco
Does a kernel MMD test catch the same wrong MuJoCo digital twins as a classifier two-sample test? Stylised MuJoCo pendulum: boosted-tree C2ST power 1.00 at n = 200 for Euler at 0.02 s and a noise-shape error where MMD stays near 0.05; MMD on second differences catches a +25% noise-level error; friction errors undetected by all at n <= 200.
What it found
A MuJoCo hinge pendulum with Student-t sensor noise is the ‘real’ system; twins change the integrator, friction or noise. Rejection uses the 95th percentile of the exact twin’s own statistic. Boosted-tree C2ST power is 1.00 at n = 200 for semi-implicit Euler at 0.02 s and 0.90 at 0.01 s, and 1.00 for a Gaussian-vs-Student-t noise-shape error, where MMD (raw or on second differences) stays at 0.00–0.07. MMD on second differences does catch a +25% noise-level error (0.95 at n = 100). A 10–20% friction error is invisible to all three at n ≤ 200. Limitations: one toy system, 20 C2ST replicates, two sample sizes. Stylised.
Paper, code and results
More in Adversarial human imitation
- Does a digital twin that a classifier can tell apart from the real system also make a worse controller?Does a digital twin that a classifier can tell apart from the real system also make a worse controller? Stylised pendulum: 6 of 15 wrong twins have zero controller regret at C2ST AUROC 0.58 to 0.999; the worst regret (3.8%) comes from a twin with no sensor noise and zero paired error. Preliminary, simulated only.
- On a swing-up task where friction matters, does a digital twin's detectability predict what tuning on it costs?On a swing-up task where friction matters, does a twin's detectability predict what tuning on it costs? Stylised pendulum: only the two gravity-error twins cost much (3.7% and 8.3% extra real cost); 13 others cost under 0.3% at C2ST AUROC 0.51 to 1.0. Preliminary, simulated only.
- Does a finer controller grid change which wrong digital twins cost something on a swing-up task?Does a finer controller grid change which wrong pendulum twins cost something on the swing-up task? No: the same two gravity-error twins cost 3.8% and 6.1%, the rest under 0.3%; detectability vs regret ρ = 0.38 (p = 0.16). Preliminary, simulated only.
- Does a wider gain grid, or a controller that uses the twin's gravity, change which wrong digital twins are costly?Does a wider gain grid, or a controller that uses the twin's gravity, change which wrong pendulum twins are costly? Gravity errors stay the costly ones; giving the controller the twin's g/L made g/L −20% regret 13 times larger (0.013 to 0.177). Preliminary, simulated only.
- Does domain randomization over a wrong digital twin cut the controller regret of tuning on one wrong twin?Does tuning a controller on a spread of randomized twins cut the regret of tuning on one wrong twin? Stylised pendulum: it brings the no-noise twin's regret from 0.0098 to 0.00006, but a very wide range raises a doubled-noise twin's worst-seed regret 7.5-fold. Preliminary, no LLM.
- Does randomizing only the sensor-noise level of a wrong digital twin keep domain randomization's rescue without its blow-up?Does randomizing only the noise level of a wrong twin keep domain randomization's rescue without its blow-up? Stylised pendulum: the no-noise twin's regret goes 0.0098 to 0 and stays small at a wide range (worst seed 0.0015 vs 0.042 for all-four), but friction and gravity errors are not helped. Preliminary, no LLM.
All 41 Adversarial human imitation projects →
Other research topics
Proper scoring & elicitation28Effort & peer prediction9Verification games & slashing19Detection & tolerances14Markets as routers13Prediction-market design9Paying contributors8Decentralised training systems23Digital twins & sim-to-real164LLM chip design loops40Robot manufacturing loops53Ambient AI assistant46Self-replicating manufacturing34Simulation-first design64Engineered clothing45Task-specific AI accelerators40Algorithmic chip-design loops42Open-source purpose-built robots44The software-engineering loop43Adversarial human imitation41Open-source residential construction41Autonomous multi-angle capture34Sensing, imaging and 3D31Energy supply and dense energy storage39Robot societies among people31Agentic co-design of low-cost robots27Laundry-hanging robots25Machine learning on qualitative data23Robots on the building site27Civilizations & group psychology1History & human sciences1
Produced with AI assistance (Claude). Results are from the code in this repository and are simulated or stylised unless the abstract says otherwise; nothing here implies affiliation with or endorsement by any lab, researcher or company cited. MIT licensed.