Is the overflow twin's lead over black-box models on a real two-tank rig statistically real? A 0.13 V RMSE gap over ARX 4/4 with a block-bootstrap 95% interval of 0.06 to 0.20 V

What it found

Puts an uncertainty interval on the headline of free-tanktwin: the 8-parameter overflow twin's held-out RMSE (0.528 V) against linear ARX 4/4 (0.655 V) on the real Cascaded Tanks measurements (Schoukens et al. 2016). Uses a paired circular moving-block bootstrap over the 974 test samples (4,000 resamples) and sweeps the block length from 1 to 200 because the test errors are autocorrelated (integrated autocorrelation time about 41 samples). The gap is 0.126 V with a 95% interval of 0.060 to 0.200 V at block 25 and 0.061 to 0.194 V at block 50; no interval at any block length includes zero, and the ordinary iid bootstrap (0.104 to 0.150 V) is about three times too narrow. Only test-trace sampling is quantified, not fit variance or a new rig, and 974 samples give roughly 24 effective blocks, so the 95% is nominal. Data are measured predictions committed by free-tanktwin; no input is invented.

A figure from its own output

Paper, code and results

• README and method
• Experiments and results
• Tests

More in Digital twins & sim-to-real

• Does the overflow twin's lead over ARX survive refitting on a real two-tank rig? A 0.13 V gap in 100% of refits with 87.5% of the training data, but 0.07 V and 84 to 90% of refits with 50 to 75%Does the overflow twin's lead over ARX survive refitting on a real two-tank rig? A 0.13 V gap in 100% of refits with 87.5% of the training data, but 0.07 V and 84 to 90% of refits with 50 to 75%. Overlapping windows, fixed optimiser start.
• Does a physics digital twin beat black-box models on a real two-tank rig? 0.53 V test RMSE with 8 parameters vs 0.66 V for the best black boxDoes a physics digital twin beat black-box models on a real two-tank rig? An 8-parameter overflow twin gets 0.53 V test RMSE against 0.66 V for the best black box (9-parameter ARX); quadratic NARX models diverge in free run. One rig, one split, no confidence interval.
• Can a 22-DOF home robot be printed in PLA around one $15 servo, and which of its twin's simplifications matter for learning to reach and pick?A 3D-printable 22-DOF bimanual mobile manipulator (two 7-DOF arms with geared elbows and parallel grippers, head, lift, omni base) on one $15 servo, $627–948 in parts, with a MuJoCo twin generated from the same parameters as the CAD: reach and pick policies trained on the exact twin ignore servo, friction and mass errors but fail at one 40 ms step of latency (reach 99→29%, pick 91→41%); randomisation fixes reaching almost for free but costs picking 14 points; an interference sweep over the printed solids caught an elbow range the twin allowed but the parts could not reach. Simulated only, not yet built.
• How few real trials certify a robot policy when a biased simulator supplies the rest?Research programme (milestones 3 to 6 logged): valid, bias-agnostic sim-to-real evaluation of robot policies, consolidating about 130 twin projects into one question, with 13 checked references, hypotheses H1 to H6 and CoRL as the target venue
• A stereo twin that adds Gaussian noise to depth gets the median right: what does it get wrong about over-range errors, safety margins and averaging compared with real disparity-space noise?Gaussian-depth-noise stereo twin vs real disparity-space noise: median exact, gated mean biased by ≈s², over-range errors up to 191× more frequent at 40 m, a twin 3σ margin exceeded 10× as often, mean-depth averaging floors while averaging disparity converges (negative result at the 60 m range floor). Stylised.
• A rigid-body twin of a cart carrying liquid predicts no slosh: how wrong is that, what does a slosh-safe move cost, and what does a stale frequency calibration do to it?A rigid twin of a cart carrying liquid predicts zero slosh: minimum-time move leaves 10.7 cm (10× tolerance), slosh-safe move is 55% slower, and a full-tank frequency calibration fails at low fill (28 s vs 8.7 s recalibrated; tuning-free bound 24.7 s).

All 164 Digital twins & sim-to-real projects →

Other research topics

Proper scoring & elicitation28Effort & peer prediction9Verification games & slashing19Detection & tolerances14Markets as routers13Prediction-market design9Paying contributors8Decentralised training systems23Digital twins & sim-to-real164LLM chip design loops40Robot manufacturing loops53Ambient AI assistant46Self-replicating manufacturing34Simulation-first design64Engineered clothing45Task-specific AI accelerators40Algorithmic chip-design loops42Open-source purpose-built robots44The software-engineering loop43Adversarial human imitation41Open-source residential construction41Autonomous multi-angle capture34Sensing, imaging and 3D31Energy supply and dense energy storage39Robot societies among people31Agentic co-design of low-cost robots27Laundry-hanging robots25Machine learning on qualitative data23Robots on the building site27Civilizations & group psychology1History & human sciences1

Produced with AI assistance (Claude). Results are from the code in this repository and are simulated or stylised unless the abstract says otherwise; nothing here implies affiliation with or endorsement by any lab, researcher or company cited. MIT licensed.