Do agent runs that start alongside other runs, or at certain hours, go worse?

What it found

Joins 660 run records that have a start time (1 to 19 October 2026) to how many other runs started within 5, 10 or 30 minutes. Raw, crowded starts look better (error recorded in 9.8% against 21.0%), but that is a day effect: 13 to 15 October had 47–50% error records and few concurrent starts, 16 October on 6–9% and many. Shuffling the crowded label within calendar days finds no detectable difference (12 tests, one p = 0.018 on a 50-run group, not counted), and six-hour blocks of the day show nothing. Preliminary and low power: association only, errors is self-reported, and duplicate work, the cost swe-schedule models, is not in the records.

A figure from its own output

Paper, code and results

• README and method
• Experiments and results
• Code
• Tests

More in The software-engineering loop

• Do parallel review sessions duplicate each other?Do parallel review sessions duplicate each other? 6 of 35 review-track PRs (17.1%, Wilson 8.1–32.7%) are the second of a pair merged 0.4 to 2.5 minutes apart, and 5 of 6 pairs edited the same generated files; cost unmeasured (3 of 35 priced). Preliminary.
• Do the agent run records match git? Does every landed session leave one?Do the agent run records match git? All 522 records naming a pull request in this repository name one that landed, but only 79.8% of sessions landing commits from 13 October have a record (ventures 79.2%, research 92.2%; preliminary, seven days).
• Why do landed ventures sessions leave fewer run records than research sessions?Why do landed ventures sessions leave fewer run records? 79.6% against research 92.2%; scope, size and kind explain nothing, the ventures sub-prompts alone omit the record step, and coverage rose from 53% to 84–89% in a week (preliminary).
• How often does this portfolio report a negative or mixed result?How often does this portfolio report a negative or mixed result? In an 80-summary hand-labelled sample, 59% of the summaries that test a claim against an alternative (43–73%) report a negative or mixed result (preliminary, one labeller).
• Do later projects build on earlier ones?Do later projects build on earlier ones? A reference graph of 723 READMEs: 61.7% of projects are named by at least one later project (Wilson 58% to 65%), inbound counts are very uneven (Gini 0.69) and 71% of references stay within a track; naming is an upper bound on reuse.
• Does an Opus agent run cost more than a Sonnet run on the same kind of work? 6x the cost, 3x the tool calls, 1.4x per self-estimated human hourDoes an Opus agent run cost more than a Sonnet run on the same kind of work? Matched on date and track, 6.3× the cost (CI 4.7–8.0), 3.2× the tool calls and 2.0× per call; 1.4× per self-estimated human hour. 46 Opus vs 958 Sonnet runs, preliminary.

All 43 The software-engineering loop projects →

Other research topics

Proper scoring & elicitation28Effort & peer prediction9Verification games & slashing19Detection & tolerances14Markets as routers13Prediction-market design9Paying contributors8Decentralised training systems23Digital twins & sim-to-real164LLM chip design loops40Robot manufacturing loops53Ambient AI assistant46Self-replicating manufacturing34Simulation-first design64Engineered clothing45Task-specific AI accelerators40Algorithmic chip-design loops42Open-source purpose-built robots44The software-engineering loop43Adversarial human imitation41Open-source residential construction41Autonomous multi-angle capture34Sensing, imaging and 3D31Energy supply and dense energy storage39Robot societies among people31Agentic co-design of low-cost robots27Laundry-hanging robots25Machine learning on qualitative data23Robots on the building site27Civilizations & group psychology1History & human sciences1

Produced with AI assistance (Claude). Results are from the code in this repository and are simulated or stylised unless the abstract says otherwise; nothing here implies affiliation with or endorsement by any lab, researcher or company cited. MIT licensed.