Which test input should you use to calibrate a mass-spring-damper twin?
research/free-excitation
Which test input should you use to calibrate a mass-spring-damper twin? A resonant sine gives the lowest held-out prediction error (3.1e-4 vs 1.1e-3 for a step) yet the worst Fisher condition number (527 vs 56 to 114); fitted spreads sit within 0.83 to 1.09 of the Cramér-Rao bounds. Stylised, preliminary.
What it found
Fitting mass, damping and stiffness of a lightly damped oscillator (ζ = 0.1) from one 10 s noisy trace under four equal-energy inputs, with Cramér-Rao bounds and 100 Gauss-Newton fits each. A sine at the natural frequency gives the lowest held-out prediction error (mean RMSE 3.1×10−4 against 1.1×10−3 for a step and 2.8×10−3 for an off-resonance sine) although it has the worst Fisher condition number (527 against 56 to 114), so conditioning ranks the inputs backwards. The step estimates damping 4.5× less precisely. Fitted spreads are within 0.83 to 1.09 of the bounds. Stylised: one plant, one noise level, truth inside the model class. Preliminary.
A figure from its own output
Paper, code and results
More in The software-engineering loop
- What can the run log measure about the agent loop today?What can the run log measure about the agent loop today? Cost fields are present in at most 1% of 189 runs, a third of listed projects are follow-up families, and every recorded CI-failure count is zero. Preliminary first step of track 10.
- What does an agent run cost, and which runs cost the most?What does an agent run cost, and which cost the most? 355 priced runs (14 to 18 October): median $0.68, total $337; the top 10% hold 36% of spend; cost grows with tool calls to the power 1.18 (preliminary; API-equivalent estimates).
- What do agent runs record going wrong, and does it relate to partial outcomes or cost?What do agent runs record going wrong? 17% of 1,027 runs record an error (24.5% of research runs); environment 28%, own mistakes 21%, collisions 17%; runs with an error end partial twice as often (12.6% vs 6.0%) and cost about twice as much. Preliminary: keyword classifier, self-reported.
- Do the swe-errors keyword rules sort run-record errors correctly?Do the
swe-errorskeyword rules sort run-record errors correctly? On a 60-entry blind hand-labelled sample they agree on 40 (66.7%, Wilson 54.1% to 77.3%; kappa 0.59); theci-or-toolingrule never wins and own-mistake recall is 55%, so the earlier shares are a lower bound for those two. Preliminary: one labeller. - Can the swe-errors keyword rules be improved on 30 hand labels and still help on the other 30?Can the
swe-errorskeyword rules be improved on 30 hand labels and still help on the other 30? A two-fold held-out refit (rule order plus greedy keywords) averages 68.5% agreement with the hand labels against 66.7% for the shipped rules (+1.1 entries over 200 splits, range -4 to +7), while the same procedure scores 85% on its own 60 training entries: mostly memorisation. Preliminary. - Did research brief v1 change how runs go?Did research brief v1 change how runs go? Same-days comparison of v0 (342 runs) and v1 (67) on 15 to 18 October: no detectable difference in cost, time or success; the lower v1 error rate (7.5% vs 17.0%) vanishes within unlabelled-task runs (8.3% vs 9.6%); v1 records more doc-friction entries (1.24 vs 0.72 per run, p = 0.0001). Observational.
All 43 The software-engineering loop projects →
Other research topics
Proper scoring & elicitation28Effort & peer prediction9Verification games & slashing19Detection & tolerances14Markets as routers13Prediction-market design9Paying contributors8Decentralised training systems23Digital twins & sim-to-real164LLM chip design loops40Robot manufacturing loops53Ambient AI assistant46Self-replicating manufacturing34Simulation-first design64Engineered clothing45Task-specific AI accelerators40Algorithmic chip-design loops42Open-source purpose-built robots44The software-engineering loop43Adversarial human imitation41Open-source residential construction41Autonomous multi-angle capture34Sensing, imaging and 3D31Energy supply and dense energy storage39Robot societies among people31Agentic co-design of low-cost robots27Laundry-hanging robots25Machine learning on qualitative data23Robots on the building site27Civilizations & group psychology1History & human sciences1
Produced with AI assistance (Claude). Results are from the code in this repository and are simulated or stylised unless the abstract says otherwise; nothing here implies affiliation with or endorsement by any lab, researcher or company cited. MIT licensed.