How many real trajectories does a classifier test need to flag a wrong digital twin, and are the twins it flags the ones that cost a controller?

What it found

On the stylised pendulum of mimic-twinregret, a boosted-tree C2ST with n real and n twin trajectories flags a twin when its AUROC beats the 95th percentile of the exact-twin null at the same n (20 replicates per cell, n = 10 to 160). The six twins whose controller regret is at least 0.001 are all flagged in at least 80% of replicates by n = 80 (by 40 for the four noise-level and noise-shape twins), so the test does not miss costly twins at that size. It also flags three twins that cost nothing (g/L −20%, noise ×0.5, noise ×0.75) by n = 40–80, while Coulomb −10% and −20% stay unflagged at 160. Spearman correlation of power at 80 with regret is 0.52 (p = 0.048, 15 twins). A flag is a cheap alarm, not a verdict on usefulness. Limits: one system, one task, 20 replicates, a null threshold that is degenerate at n ≤ 20. Stylised, simulated, no LLM.

A figure from its own output

Paper, code and results

• README and method
• Experiments and results
• Tests

More in Adversarial human imitation

• Does a second turn make a plainly prompted LLM easier to spot in dialogue?Does a second turn make a plainly prompted LLM easier to spot in dialogue? A prompt-free n-gram detector gets AUROC 0.93 on one imitated user turn and 0.98 on two, on 400 OpenAssistant threads with a 0.5B imitator. Stylised baseline, no prompt optimised.
• Can an imitator that changes its own code fool a human-vs-bot cursor detector, and does shared feedback help either side?Can an imitator that changes its own code fool a human-vs-bot cursor detector, and does shared feedback help? On DUX cursor strokes of 46 people, a held-out reference detector catches every final imitator at AUROC 0.99 to 1.00 in all four feedback conditions; two-way sharing gave the strongest detectors in cross-play (0.81 vs 0.70 blind, interval includes 0), and an 8x larger search stops at 0.88, the level of replaying real strokes. Preliminary.
• Can a generator imitate a person's webcam emotion signals well enough to fool a detector?Can a generator imitate a person's webcam emotion signals well enough to fool a detector? On DUX Affectiva scores of 46 people, every fixed generator and every loop-tuned imitator is caught at AUROC 0.997 or higher by a detector the loop never saw; only replaying real windows gets close to chance (0.61). Preliminary.
• Does giving a webcam-signal generator real heavy tails and bursts fool a detector?Does giving a webcam-signal generator real heavy tails and bursts fool a detector? No: copula and quiet/burst generators still separate from held-out people at AUROC 1.000 (8 seeds); restoring per-window level and spread brings level and spread to near chance, but extremes (0.995) still give it away; copying a real window's value set (G6) puts those at chance too, yet all-groups AUROC stays 0.998 on temporal structure.
• Is the prompted detector's inversion (AUROC below 0.5 on machine text) a model-size effect?Does a bigger Qwen make a better human-vs-machine detector? Partly: plain-prompt AUROC rises from 0.278 (0.5B) to 0.329 (1.5B), but 1.5B is still below chance in 84% of family-by-prompt cells.
• Can a small open code model write a cursor-stroke generator that fools a human-vs-bot detector?A 1.5B code model rewrites the cursor generator against the mimic-cursor detector: only 25 to 29% of its candidates run, and the reference detector stays at AUROC 1.000, tied with the component search at the same 24-candidate budget (one seed, preliminary negative).

All 41 Adversarial human imitation projects →

Other research topics

Proper scoring & elicitation28Effort & peer prediction9Verification games & slashing19Detection & tolerances14Markets as routers13Prediction-market design9Paying contributors8Decentralised training systems23Digital twins & sim-to-real164LLM chip design loops40Robot manufacturing loops53Ambient AI assistant46Self-replicating manufacturing34Simulation-first design64Engineered clothing45Task-specific AI accelerators40Algorithmic chip-design loops42Open-source purpose-built robots44The software-engineering loop43Adversarial human imitation41Open-source residential construction41Autonomous multi-angle capture34Sensing, imaging and 3D31Energy supply and dense energy storage39Robot societies among people31Agentic co-design of low-cost robots27Laundry-hanging robots25Machine learning on qualitative data23Robots on the building site27Civilizations & group psychology1History & human sciences1

Produced with AI assistance (Claude). Results are from the code in this repository and are simulated or stylised unless the abstract says otherwise; nothing here implies affiliation with or endorsement by any lab, researcher or company cited. MIT licensed.