Would a spend cap have stopped your runaway LLM bill?

1. Replay a usage log through a policy

Rows need a model and token counts in Anthropic (input_tokens, output_tokens, cache_read_input_tokens) or OpenAI (prompt_tokens, completion_tokens) form, nested under usage or flat. Optional: created_at, user, feature, run_id/trace_id, and prompt_hash (without it a replay cannot see repeats). Prices come from the policy: put in your provider's current rates.

2. What the policy would have done

Breaker trips and budget alerts

First refused calls

Arithmetic on your log and the prices in your policy: not a forecast and not your invoice.

3. Enforce it: proxy or library

Proxy (Node 18+, no dependencies). Change the SDK's base URL and every call passes the guard:

node cli.mjs init guard-policy.json        # then put in your models and current prices
node cli.mjs check-policy guard-policy.json  # summary + gaps (no global cap? no breakers?)
node cli.mjs proxy guard-policy.json --port 8787

ANTHROPIC_BASE_URL=http://127.0.0.1:8787/anthropic
OPENAI_BASE_URL=http://127.0.0.1:8787/openai/v1
# optional headers per request: x-guard-user, x-guard-feature, x-guard-run

A refused call gets the provider's own error shape: HTTP 402 when a budget is spent (SDKs do not retry it) and 429 with Retry-After while a breaker is open. Streams are passed through byte for byte and settled from the usage they report.

Library, around any SDK call:

import { createGuard } from './guard.mjs';
const guard = createGuard(policy, { onEvent: (e) => e.type === 'alert' && notify(e) });

const text = await guard.guarded(
  { model, maxOutputTokens: 1024, input: messages, scope: { user: userId, run: runId } },
  async (d) => {
    const r = await client.messages.create({ model, max_tokens: d.maxOutputTokens, messages });
    return { result: r, usage: { input: r.usage.input_tokens, output: r.usage.output_tokens } };
  });

Read how to set a hard spend limit on OpenAI and Anthropic API calls for what each control catches and where provider-side limits still matter.

Questions

How do I put a hard limit on my OpenAI or Anthropic API spend?

Set the provider's own organisation or project limit as the backstop, then put a guard in front of your calls that refuses a request before it is sent when it would cross a budget. LLM Spend Guard does that as a local proxy (change the SDK base URL) or as a library call around your SDK call, with budgets per user, feature, key and agent run.

How does it stop concurrent calls overshooting a budget?

Each call holds its worst-case cost (a high estimate of its input plus its full output cap) against every budget before it is sent, like a card hold. Concurrent calls cannot all fit into the same remaining dollar. When the response reports real usage, the actual cost replaces the hold.

What is a runaway loop breaker?

An agent that keeps retrying the same tool call, or a bug that calls the model in a tight loop, can spend a month's budget in minutes. A breaker watches a rolling window and opens when one run sends the same request too many times, or calls or spend per minute pass a limit; while open it refuses calls for a cooldown, and the proxy answers HTTP 429 with Retry-After.

Is my usage log uploaded?

No. The simulator on this page replays the log you paste in your own browser and never sends it anywhere. The proxy runs on your machine or your server, and only talks to the provider you point it at.