How many real steps does a twin-pretrained policy save, and which twin errors does fine-tuning inherit?

research/finetune-twin

Fine-tuning a twin-pretrained policy on the real plant: exact SGD risk, a twin start saves 64–67% of real steps at 0.6× the cold error but is negative transfer beyond 1×, and twin error decays only as b²/(4eηk) (flat directions inherited: 13 vs 1,541 steps at equal initial loss).

8 tests

What it found

On a quadratic real loss with SGD noise, the expected loss after k real steps from a twin start is exact (matched to Monte Carlo within 0.3%). A twin start with 0.6× the cold-start error saves 64–67% of the steps to every target from 8× to 1.2× the SGD floor; at equal error it saves nothing and at 1.5× it costs 138 extra steps (negative transfer). For any curvature spectrum the loss left by a twin error of norm b is at most b²/(4eηk), attained by the direction λ_*=1/(2ηk), so twin error decays as 1/k, not geometrically. At the same initial loss, an error in the stiffest direction needs 13 steps to reach 2× the floor and one in the flattest 1,541. Stylised: fixed offset, isotropic noise, no robot data.

A figure from its own output

Figure produced by research/finetune-twin: transfer_saving — finetune-twin/experiments/transfer_saving.svg, written by the project's own experiment code.

Paper, code and results

More in Digital twins & sim-to-real

How far ahead should a controller planned in a twin with the wrong plant pole look?How far ahead a controller planned in a wrong-pole twin should look: exact real cost per horizon is non-monotone when the twin overstates the pole (1.81 at H=2 vs 3.97 at H=60, optimum 1.48), the twin's own claim ranks horizons backwards, exact two-step stability edge, and a Riccati terminal cost removes the effect.
How should cheap biased twin samples be weighted against scarce real ones?Mixing cheap biased twin samples with scarce real ones: exact optimal weight, twin worth capped at σ²/δ² real samples, equal pooling helps iff δ²<a+b, and estimating the bias from data is costly (plug-in worse than real-only for large bias).
How much load can a shared lab instrument carry when its twin assumes exponential service?How much load a shared lab instrument carries when its twin assumes exponential service: exact Pollaczek–Khinchine wait off by (1+c²)/2, twin admits ρ=0.667 vs real 0.444 (c²=4) or 0.133 (c²=25), tails 11.7× worse, infinite mean wait for Pareto index < 2, and a plug-in c² repair that violates the SLA 70–89% of the time.
How much smoothing does a differentiable twin of a hard contact event need, and what does it cost?Smoothing a hard contact edge to make a twin trainable: real gradient is zero, twin ascent is trapped below τ_c=0.105 and misses the edge above R/(8cm), annealing crosses the trap, and the overshoot ≈τ ln(R/(2cmτ)) is an implicit safety margin capped at 0.168.
How many redundant sensors does a twin with independent errors say a robot needs?How many redundant sensors a twin with independent errors says a robot needs: exact equicorrelated fused variance, n_eff < 1/ρ and a floor σ√ρ, twin says 16 sensors vs real 30 (ρ=0.03) or none (ρ≥0.0625), 3σ gate false-alarms 8.6× too often, and a plug-in ρ repair that misses the target about half the time.
How much is one long digital-twin run worth when its output is serially correlated?How much one long digital-twin run is worth when its output is autocorrelated: exact AR(1) run-mean variance, M/M/1 wait inflation 9.7×–363× (ρ=0.5–0.9), naive 95% intervals cover 9–53%, false SLA certification 26% vs 4.8% with batch means, and batch means fail when batches are shorter than the correlation time.

All 164 Digital twins & sim-to-real projects →

Other research topics

Proper scoring & elicitation28 Effort & peer prediction9 Verification games & slashing19 Detection & tolerances14 Markets as routers13 Prediction-market design9 Paying contributors8 Decentralised training systems23 Digital twins & sim-to-real164 LLM chip design loops40 Robot manufacturing loops53 Ambient AI assistant46 Self-replicating manufacturing34 Simulation-first design64 Engineered clothing45 Task-specific AI accelerators40 Algorithmic chip-design loops42 Open-source purpose-built robots44 The software-engineering loop43 Adversarial human imitation41 Open-source residential construction41 Autonomous multi-angle capture34 Sensing, imaging and 3D31 Energy supply and dense energy storage39 Robot societies among people31 Agentic co-design of low-cost robots27 Laundry-hanging robots25 Machine learning on qualitative data23 Robots on the building site27 Civilizations & group psychology1 History & human sciences1

Produced with AI assistance (Claude). Results are from the code in this repository and are simulated or stylised unless the abstract says otherwise; nothing here implies affiliation with or endorsement by any lab, researcher or company cited. MIT licensed.