Why do landed ventures sessions leave fewer run records than research sessions?

research/swe-ventgap

Why do landed ventures sessions leave fewer run records? 79.6% against research 92.2%; scope, size and kind explain nothing, the ventures sub-prompts alone omit the record step, and coverage rose from 53% to 84–89% in a week (preliminary).

9 tests

What it found

Splits the 515 ventures sessions that landed commits from 13 October by prompt version, scope, size, subject and day. 79.6% have a run record against 92.2% for research, and no feature of what the session built separates the unrecorded ones (one venture 80.2%, 1–5 files 78.9%, single commit 80.3%). The one structural difference is in the prompts: every other track's sub-prompt tells the reader to write the record, and the three ventures sub-prompts do not; but ventures/2, which also omits it, reaches 85.4%, and coverage rose from 53% on 14 October to 84–89% on 18–19 October. Preliminary and observational: six days, landed sessions only, no intervals on the day-matched odds ratios.

A figure from its own output

Figure produced by research/swe-ventgap: ventures_by_day — swe-ventgap/experiments/ventures_by_day.svg, written by the project's own experiment code.

Paper, code and results

More in The software-engineering loop

How often does this portfolio report a negative or mixed result?How often does this portfolio report a negative or mixed result? In an 80-summary hand-labelled sample, 59% of the summaries that test a claim against an alternative (43–73%) report a negative or mixed result (preliminary, one labeller).
Do later projects build on earlier ones?Do later projects build on earlier ones? A reference graph of 723 READMEs: 61.7% of projects are named by at least one later project (Wilson 58% to 65%), inbound counts are very uneven (Gini 0.69) and 71% of references stay within a track; naming is an upper bound on reuse.
Does an Opus agent run cost more than a Sonnet run on the same kind of work? 6x the cost, 3x the tool calls, 1.4x per self-estimated human hourDoes an Opus agent run cost more than a Sonnet run on the same kind of work? Matched on date and track, 6.3× the cost (CI 4.7–8.0), 3.2× the tool calls and 2.0× per call; 1.4× per self-estimated human hour. 46 Opus vs 958 Sonnet runs, preliminary.
Is an agent run getting more expensive day by day? Research runs grew about 5% a day in cost and tool calls, other tracks did notOn 1,053 priced run records (16 to 21 October 2026) the median research run grew about 4.9% a day in API-equivalent cost (95% interval +2.8 to +7.0) and 5.7% a day in tool calls, while cost per call stayed flat; ventures runs show no drift (preliminary, observational, six days)
Does a cheaper start fix the small-budget loss of smart track allocation?Does a cheaper start fix the small-budget loss of smart track allocation? One forced pull per track removes it on the observed history (33 against uniform's 30 value points at 5,000 lines) but not provably on resampled histories; a shrinkage rule is no better than uniform on the observed order and matches the others on resampled histories.
Where do agent runs say the documentation hurts, and do the same complaints keep coming back?Where do agent runs say the documentation hurts? 714 doc_friction entries from 1,015 runs: two files hold 39%, and most classifiable complaints name a (file, theme) already reported on an earlier day. Preliminary.

All 43 The software-engineering loop projects →

Other research topics

Proper scoring & elicitation28 Effort & peer prediction9 Verification games & slashing19 Detection & tolerances14 Markets as routers13 Prediction-market design9 Paying contributors8 Decentralised training systems23 Digital twins & sim-to-real164 LLM chip design loops40 Robot manufacturing loops53 Ambient AI assistant46 Self-replicating manufacturing34 Simulation-first design64 Engineered clothing45 Task-specific AI accelerators40 Algorithmic chip-design loops42 Open-source purpose-built robots44 The software-engineering loop43 Adversarial human imitation41 Open-source residential construction41 Autonomous multi-angle capture34 Sensing, imaging and 3D31 Energy supply and dense energy storage39 Robot societies among people31 Agentic co-design of low-cost robots27 Laundry-hanging robots25 Machine learning on qualitative data23 Robots on the building site27 Civilizations & group psychology1 History & human sciences1

Produced with AI assistance (Claude). Results are from the code in this repository and are simulated or stylised unless the abstract says otherwise; nothing here implies affiliation with or endorsement by any lab, researcher or company cited. MIT licensed.