How much can a provider steal by serving a cheaper model?

What it found

A decentralised inference provider substitutes a cheaper model on a random fraction p of queries; a verifier re-scores a fraction f under an anytime-valid mixture e-process. Detection delay scales as 1/(f p² χ²), so the profitable attack is a slow leak: undetected savings grow as √N rather than N (measured ×7.75 for ×64 queries). Closed-form audit rate to cap theft, cost-optimal check rate (hK/2Nc)^2/3, and a substitute close enough to the claimed model that even checking every response cannot cap the theft. Seeded Monte Carlo. Stylised.

A figure from its own output

Paper, code and results

• README and method
• White paper
• Experiments and results
• Code
• Tests

More in Detection & tolerances

• Can you slash while watching the data stream?Anytime-valid slashing of miscalibrated verifiers: peeking z-test falsely slashes 37% of honest verifiers (nominal 5%), an e-process holds ≤α by Ville, detection delay ≈ ln(1/α)/KL (within 12%), q-free mixture costs ×1.36.
• Can two verifiers be compared while the data streams in?Anytime-valid duels between two verifiers: the optimal betting e-process on the Brier difference grows at exactly KL(q‖(rA+rB)/2), a peeking z-test falsely declares a winner 19–38% of the time at nominal 5%, a bet mixture stays ≤3.8% and costs 15–41% more decisions.
• Which of n verifiers is best, decided at any time?Anytime-valid best-of-n verifier identification: delay set by the rival whose report midpoint sits nearest q (the best opposite-side rival, not the runner-up), mixture within 1.35–1.60× of oracle, elimination 4–5× cheaper, peeking declares a false best in 59% of ties vs 1.6%.
• How much can Byzantine verifiers move an aggregate?Byzantine-robust aggregation of verifier reports: mean bias βB unbounded, median bias σΦ⁻¹(1/(2(1−β))) independent of B, trimmed-mean closed form for τ≥β verified by simulation.
• Can a verifier tell from old-task losses that a continual-learning job skipped or shortchanged tasks?Auditing skipped or partial continual-learning training from old-task losses: exact loss law (ρ^{t−1}[(1−2m₁+m₂)(r/d)λᵏ+(r/d)ρ(ρᵏ−λᵏ)]), a skipped task is ≥ d/(d−r)× worse but only clearly so inside a lag window, detecting a skipper takes ~0.7/s² jobs at any lag, a 10% step shortfall needs ~9600.
• Is an interference-free or mean-noise-rise radar twin right when robots' radars interfere with each other?Interference-free or mean-noise-rise radar twin vs intermittent mutual radar interference between robots: exact closed-form per-frame Pd, real Pd plateaus at 0.656 (no-hit probability), mean-rise twin 9% optimistic at Pd ≥ 0.9 yet badly pessimistic for hit-tolerant missions, geometry knowledge helps 9-of-10 missions (Brier 0.089 vs 0.101). Stylised.

All 14 Detection & tolerances projects →

Other research topics

Proper scoring & elicitation28Effort & peer prediction9Verification games & slashing19Detection & tolerances14Markets as routers13Prediction-market design9Paying contributors8Decentralised training systems23Digital twins & sim-to-real164LLM chip design loops40Robot manufacturing loops53Ambient AI assistant46Self-replicating manufacturing34Simulation-first design64Engineered clothing45Task-specific AI accelerators40Algorithmic chip-design loops42Open-source purpose-built robots44The software-engineering loop43Adversarial human imitation41Open-source residential construction41Autonomous multi-angle capture34Sensing, imaging and 3D31Energy supply and dense energy storage39Robot societies among people31Agentic co-design of low-cost robots27Laundry-hanging robots25Machine learning on qualitative data23Robots on the building site27Civilizations & group psychology1History & human sciences1

Produced with AI assistance (Claude). Results are from the code in this repository and are simulated or stylised unless the abstract says otherwise; nothing here implies affiliation with or endorsement by any lab, researcher or company cited. MIT licensed.