Research topic
Paying contributors
Buying compute, data and model improvements: holdout markets, Shapley payments, contests and races.
8 projects, each with its own page, code and tests. See them on the full research index.
research/holdout-market · 4 tests
Paying for model improvements against a hidden holdout: telescoping budget, a payment-feedback overfitting attack (paid 0.15 while true loss worsens 0.37), and the Ladder fix with its measured cost.
Abstract
A model-improvement market paying holdout-loss decreases: the budget telescopes, but naive payment feedback lets a data-less adversary be paid while the published model's true loss worsens; a Ladder-style threshold removes the attack at a measured cost to honest small contributors. Stylised.
research/data-valuation-replication · 9 tests
Shapley and leave-one-out payments under replication: Shapley pays every extra copy even when worthless, saturating at stand-alone value ((n+2)/2× honest), naive evaluators hand over the pot, LOO pays copies 0 but distributes 1/(n+1), dedup underpays lookalikes by 45%.
Abstract
Paying contributors of data or gradients by Shapley value rewards replication even when copies add zero information: in a Bayesian mean-estimation game the replicator's payment rises with every copy to its stand-alone value, exactly (n+2)/2 times the honest payment against n unit-precision others (5.5× at n=9). If the evaluator treats copies as independent the replicator's pot share tends to 100%; leave-one-out pays true copies exactly zero but distributes only 1/(n+1) of the pot; collapsing duplicates fixes it but underpays independent look-alikes by 45%. Exact computation checked against permutation brute force. Stylised.
research/shapley-sampling · 9 tests
Auditing sampled Shapley payments: exact estimator variances in a saturating game, permutation sampling is exactly efficient but stratified sampling has 6–8× lower variance (zero for equal contributors) at up to 7% budget error, antithetic reversal gives 1.6× lower RMSE, Hoeffding sample counts are 4–5× conservative.
Abstract
In a saturating contribution game v(S)=1−exp(−Σwi) every estimator variance is exact. Permutation sampling always pays out exactly v(N) but is noisy; size-stratified sampling has 6–8× lower variance (exactly zero for equal contributors) yet misses the pot by up to 7% in a run; reversing each permutation cuts RMSE 1.6× at no cost; Hoeffding sample counts are 4–5× larger than needed, and stratification finds the top-3 payees with about 4× fewer samples. Stylised, n≤10.
research/contest-precision · 11 tests
Winner-take-all training contests scored on a noisy holdout: effort V a_n √m/s with a_2 = a_3 = 1/(2√π) exactly, a precision ceiling V/σ² ≤ λ_max(n) beyond which no pure equilibrium exists (below the zero-rent limit), and the resulting optimal holdout size.
Abstract
In a winner-take-all contest ranked on a noisy holdout of m samples, equilibrium effort is V·an√m/s with a2=a3=1/(2√π) exactly, but the formula stops being an equilibrium once V/σ² passes a ceiling (12.35, 7.78, 6.11, 5.60, 5.80 for n=2, 3, 5, 10, 20) that sits below the zero-rent limit and falls to half of it at n=20. The organiser's optimal holdout is min(mcap, (wVan/2sκ)²): when evaluation is cheap the ceiling, not cost, binds. Quadrature checked against Monte Carlo and brute-force best replies. Stylised.
research/compute-procurement · 9 tests
Buying k units of decentralised compute by reverse auction: exact VCG payment k(k+1)/(n+1) is exactly 2× winners' cost, optimal reserve is exactly v/2, first-price equilibrium matches by revenue equivalence.
Abstract
Reverse auction for k compute units from n devices with private costs: the truthful threshold auction pays exactly k(k+1)/(n+1), twice the winners’ true cost for every n and k (also with a reserve), the buyer-optimal reserve is exactly v/2 (worth up to +1.03 utility in thin markets, nothing once v≥2), and the pay-as-bid equilibrium has closed form and matches revenue (1.3334 vs 1.3333). Stylised.
research/speed-race · 12 tests
Paying the first k of n finishers makes speed a contest: π'(1)=(1−k/n)(H_n−H_{n−k}), equilibrium speed R(n−k)(H_n−H_{n−k})/(nc), exact round time nc/((n−k)R); paying all finishers buys no speed, a budget favours winner-take-all until an accuracy floor k_min binds, and recruiting more workers barely speeds the round.
Abstract
If a round pays the first k of n finishers a prize R and workers buy speed at cost c per unit rate, the win probability’s slope at the symmetric point telescopes to (1−k/n)(Hn−Hn−k), so equilibrium speed is R(n−k)(Hn−Hn−k)/(nc) and the round time is exactly nc/((n−k)R). Paying every finisher buys no speed at all; for a fixed budget, speed alone wants winner-take-all (0.10 at k=1 versus 3.2 at k=16 for n=32), and only the statistical floor on k justifies more winners. Recruiting more workers barely helps: total speed bought saturates at kR/c. Exponential races, linear cost, symmetric risk-neutral workers; equilibrium checked on a grid.
research/race-heterogeneity · 13 tests
Paying the first k of n finishers when costs differ: proof of the π'(1) telescoping identity, exact exit rule (slow workers idle iff c_s/c_f ≥ n_f/(n_f−k), never for k ≥ n_f), cost ratio 4 excludes slow workers for k ≤ 3 of 4 fast, convex costs make headcount buy speed.
Abstract
If only the first k of n finishers are paid, a slow worker entering at a vanishing rate wins with probability rate × E X(k) of the fast workers, which gives an exact exit rule: workers with cost ratio cs/cf ≥ nf/(nf−k) stay out entirely, and paying k ≥ nf never excludes anyone (checked by best-entry payoffs at thresholds 4/3, 2, 4). At cost ratio 4 with four fast workers, k ≤ 3 shuts the slow ones out and k=4..7 readmits them (share of prizes 0.21→0.46 against a fair 0.5). The note also proves the telescoping identity behind the symmetric race and shows that with convex speed cost, headcount does buy speed. Two cost types, exponential races, grid-checked equilibria.
research/private-data-market · 8 tests
Buying private data with a telescoping log-score market: payment equals information, best Gaussian-mechanism noise in closed form, participation iff κa<1 so precision is capped at λ/(cΔ²) for any number of contributors, and early arrivals over-collect (1–22% welfare loss).
Abstract
A market pays each contributor the log-score gain of a public Gaussian belief for releasing a statistic plus Gaussian-mechanism noise, so the expected payment is exactly the information released, ½λln(1+τ/a). The best noise is a closed-form quadratic root, and participation is possible if and only if κa < 1 with κ = cΔ2/λ: precision is capped at λ/(cΔ2) however many contributors arrive, approached geometrically at rate 2σ2/(2σ2+κ), and total payout is at most ½λln(1/(κa0)). Early arrivals add too little noise (1.5–5×; 1–22% welfare loss against a planner), and admitting small data holders first lifts final precision from 28.0 to 34.2. Checked against grid search and 106-run Monte Carlo. Stylised.
Other research topics
Proper scoring & elicitation28Effort & peer prediction9Verification games & slashing19Detection & tolerances14Markets as routers13Prediction-market design9Paying contributors8Decentralised training systems23Digital twins & sim-to-real164LLM chip design loops40Robot manufacturing loops53Ambient AI assistant46Self-replicating manufacturing34Simulation-first design64Engineered clothing45Task-specific AI accelerators40Algorithmic chip-design loops42Open-source purpose-built robots44The software-engineering loop43Adversarial human imitation41Open-source residential construction41Autonomous multi-angle capture34Sensing, imaging and 3D31Energy supply and dense energy storage39Robot societies among people31Agentic co-design of low-cost robots27Laundry-hanging robots25Machine learning on qualitative data23Robots on the building site27Civilizations & group psychology1History & human sciences1
Produced with AI assistance (Claude). Results are from the code in this repository and are simulated or stylised unless the abstract says otherwise; nothing here implies affiliation with or endorsement by any lab, researcher or company cited. MIT licensed.