Reproducible operators and refereed training

What it found

Measured float32 drift across reduction orders (relative drift is Cauchy-tailed, state-level drift is a few ulps), a Merkle-committed training trace with a teacher-forced referee, and the finding that a per-step tolerance is a cumulative hidden-deviation budget while bitwise-reproducible operators remove it.

Paper, code and results

• README and method
• White paper
• Experiments and results
• Code
• Tests

More in Detection & tolerances

• How many honest runs does a slashing tolerance need?Calibrating slashing tolerances from honest runs: split-conformal false-slash law (n+1-k)/(n+1) is exact but only ~51–56% of calibration sets are ≤ α, PAC safety needs n ≥ ln δ/ln(1−α) (459 at 1%/1%), the margin is cheater hiding room (×1.26 at n=2000, Pareto-3), Gaussian plug-ins over-slash skewed drift up to 17×, pooled hardware calibration over-slashes the noisy class 4.8× and lets the quiet class hide 2.9×.
• Should drift tolerances be per layer or on the total?Per-layer vs total drift tolerances: a tolerance per layer admits L·c(L,α)σ vs z_α√L σ on the total (6.6× at L=16), the sum test ignores how a cheat is spread while the max test catches 2.7σ in one layer but lets 16σ through spread over all, and a union test caps the hidden budget at the sum test's.
• How well does Freivalds' check work in floating point?Freivalds' matrix-product check in floating point: rank-one corruption missed with probability exactly erf(t/(√2F)) (worst case for fixed norm), erf^m for m probes, Rademacher probes stuck at ½ for any size, measured float32 noise floor, and 1/erf grinding tries when the probe is hashed from the result.
• How many skipped training steps hide in the loss noise?Detecting skipped training steps from the loss alone: hidden compute is the loss curve inverted at the noise band (f* = 1−(1+(e^q−1)/r)^{−1/β}), grows as reducible loss falls, seed noise sets a floor no eval size removes, simulation matches within 1.5 points, and the loss test is worth ≈4.4 step audits at best.
• How much can a provider steal by serving a cheaper model?Detecting a provider that serves a cheaper model on a fraction of queries: mixture e-process (valid under peeking), delay ∝ 1/(f p² χ²), so the cheater's best response is to leak rarely and steal ∝ √N not N; closed-form audit rate and cost-optimal check rate (hK/2Nc)^{2/3}.
• Can you slash while watching the data stream?Anytime-valid slashing of miscalibrated verifiers: peeking z-test falsely slashes 37% of honest verifiers (nominal 5%), an e-process holds ≤α by Ville, detection delay ≈ ln(1/α)/KL (within 12%), q-free mixture costs ×1.36.

All 14 Detection & tolerances projects →

Other research topics

Proper scoring & elicitation28Effort & peer prediction9Verification games & slashing19Detection & tolerances14Markets as routers13Prediction-market design9Paying contributors8Decentralised training systems23Digital twins & sim-to-real164LLM chip design loops40Robot manufacturing loops53Ambient AI assistant46Self-replicating manufacturing34Simulation-first design64Engineered clothing45Task-specific AI accelerators40Algorithmic chip-design loops42Open-source purpose-built robots44The software-engineering loop43Adversarial human imitation41Open-source residential construction41Autonomous multi-angle capture34Sensing, imaging and 3D31Energy supply and dense energy storage39Robot societies among people31Agentic co-design of low-cost robots27Laundry-hanging robots25Machine learning on qualitative data23Robots on the building site27Civilizations & group psychology1History & human sciences1

Produced with AI assistance (Claude). Results are from the code in this repository and are simulated or stylised unless the abstract says otherwise; nothing here implies affiliation with or endorsement by any lab, researcher or company cited. MIT licensed.