Where does an agent run's cost go, and why does it grow faster than its tool calls?
research/swe-cachecost
Where does a run's cost go? On 893 run records with per-token-kind usage, cache reads are a median 45% of the API-equivalent cost and 1-hour cache writes 41%, output only 13%; the starting context (about 60,000 tokens) is re-read on every request, which makes cost grow faster than linearly in tool calls (preliminary).
What it found
Of 1,659 run records, 893 carry per-token-kind usage as well as cost and tool calls. At API-equivalent list prices, cache reads are a median 45% of a run's cost, 1-hour cache writes 41% and output 13%; fresh input is about zero. A linear context model fits the cache-read tokens per request as about 60,000 tokens at the start plus about 1,900 tokens more per request (bootstrap 95% interval 1,500 to 2,200), so at the median run (20 requests) the starting context is 76% of what each request re-reads. That fixed part, written once and re-read throughout, is why cost rises with a log-log slope of 1.16 against requests and 1.22 against tool calls rather than 2. Limitations: API-equivalent dollars from list prices, not a bill; the context model explains only 31% of the per-request variance; descriptive, from one repository's runs.
A figure from its own output
Paper, code and results
More in The software-engineering loop
- Do agent runs that add no tests end worse?Do agent runs that add no tests end worse? 84 of 621 counted run records add none; within track they end partial 4/84 against 16/537 (+2.6 points, p = 0.23, no detectable effect) and cost less, probably because they do less. Preliminary.
- Which kind of recorded deviation from the brief predicts a partial outcome?Which kind of recorded deviation predicts a partial outcome? Of 213 run records with a deviation, tooling failures end partial 22.6 points more often than runs with none (track-stratified p=0.001); skipped steps (+1.3) and switched tasks (+1.3) do not. Keyword classes, observational, preliminary.
- How complete are the agent run records, field by field?How complete are the agent run records, field by field? Of 1,362 records, cost, tokens and wall time are present on 47% (0-7% on 13 to 15 October, 61-82% since),
mergedon 4.6%, and 18.5% ofsuccessruns name no pull request (preliminary; presence, not correctness). - Does a bigger agent run buy more output?Does a bigger run buy more output? Across 157 priced run records that report adding tests, cost and tests added rank together pooled (Spearman 0.57) but weakly within research runs (0.19), and the dearest third yields the fewest tests per dollar; preliminary, tests added is a crude self-reported proxy.
- What does an agent run cost per merged pull request?What does a run cost per merged pull request? 366 priced runs joined to
main's merge history: 75% name a merged PR, $1.26 per merged run against $0.94 per run, process runs about 2.4 times research's; red team never links its PR (preliminary; API-equivalent estimates). - What does git history say about the agent loop that the run log cannot?What does git history say about the agent loop that the run log cannot? All 157 projects created since 2026-10-13 are in the log, the 217 it lacks are pre-log bulk output, and at most 3% of fully observed projects have a correction-worded follow-up commit. Preliminary second step of track 10.
All 43 The software-engineering loop projects →
Other research topics
Proper scoring & elicitation28Effort & peer prediction9Verification games & slashing19Detection & tolerances14Markets as routers13Prediction-market design9Paying contributors8Decentralised training systems23Digital twins & sim-to-real164LLM chip design loops40Robot manufacturing loops53Ambient AI assistant46Self-replicating manufacturing34Simulation-first design64Engineered clothing45Task-specific AI accelerators40Algorithmic chip-design loops42Open-source purpose-built robots44The software-engineering loop43Adversarial human imitation41Open-source residential construction41Autonomous multi-angle capture34Sensing, imaging and 3D31Energy supply and dense energy storage39Robot societies among people31Agentic co-design of low-cost robots27Laundry-hanging robots25Machine learning on qualitative data23Robots on the building site27Civilizations & group psychology1History & human sciences1
Produced with AI assistance (Claude). Results are from the code in this repository and are simulated or stylised unless the abstract says otherwise; nothing here implies affiliation with or endorsement by any lab, researcher or company cited. MIT licensed.