Do prompts that fool a prompted LLM detector also fool detectors that use no prompt?
research/mimic-looptransfer
Do the mimic-loop imitator prompts fool detectors they never saw? No: matched on length, all 26 testable prompts are still caught (n-gram AUROC 0.83 to 0.96, Binoculars-style 0.84 to 1.00); low raw scores come from text length.
What it found
Replays all 32 imitator prompts from mimic-loop (Qwen2.5-0.5B-Instruct, reddit and film reviews, 48 texts per prompt) and scores them with the mimic-contexts prompt-free detectors, refit without the loop's test items. Raw AUROCs range from 0.13 to 0.99, but they track mean generated length (Spearman +0.86 to +0.97): very short texts get a higher log-perplexity and look more human. Comparing only pairs within 1.5× in length, every prompt with enough pairs (26 of 32) is still caught, n-gram AUROC 0.83–0.96 and Binoculars-style 0.84–1.00. A prompt's paired accuracy against the loop's own prompted detector does not predict its prompt-free AUROC (Spearman −0.19 to +0.16 over all prompts, with length-matched signs negative). Preliminary: one loop seed, one 0.5B model, two families, 24 human texts per family; the detectors are trained on RAID chat-model text, not Qwen text, so some distance is generator shift.
A figure from its own output
Paper, code and results
More in Adversarial human imitation
- Can a learned Markov chain over a person's webcam frames fool a detector?Can a Markov chain over a person's frames fool a detector? No: chains with 8, 32 or 128 states, fitted on one person or on everyone, emitting state centres or real frames, are caught at AUROC 1.000 (one cell 0.999), where stitching the same real frames in 50-frame blocks scored 0.57 to 0.65.
- Does fitting a webcam-signal generator to one person fool a detector that has seen that person?Does fitting to one person help? Copying a person's own earlier windows is caught at AUROC 0.59 to 0.65 on their later windows; copying other people is 0.58 to 0.91, and 0.93 to 0.97 once the detector knows the person; a fitted parametric generator is caught at 1.000 either way.
- Does giving a Markov chain dwell times or a second-order memory let it fool a webcam-signal detector?Does giving a chain over a person's frames dwell times or second-order memory help? No: semi-Markov and second-order chains (8 or 32 states, own or pooled fit, centre or real-frame emission) are all caught at AUROC 0.999 to 1.000.
- Why is block-stitched real webcam signal still caught: the joins, or mixing people?Why is block-stitched real webcam signal still caught? With 50-frame blocks random stitching gives AUROC 0.887, one person's windows only 0.687, join-matched 0.796, both 0.588 (= whole-window replay 0.597); at 5 frames all are caught (8 seeds).
- How well do prompt-free detectors tell people from a plainly prompted LLM, and do they carry over to unseen kinds of text?How well do prompt-free detectors tell people from a plainly prompted LLM, and do they carry over to unseen kinds of text? Eight RAID families split by family: n-gram detector AUROC 0.93 and Binoculars-style 0.80 on held-out families, length at chance. Preliminary first step of track 11.
- Do the file sizes and line counts of this repository follow Benford’s law?Do this repository's file sizes follow Benford's law? Not as a whole (byte MAD 0.018, nonconforming) because templated files repeat the same size thousands of times; counting each distinct size once gives acceptable conformity (MAD 0.0068).
All 41 Adversarial human imitation projects →
Other research topics
Proper scoring & elicitation28Effort & peer prediction9Verification games & slashing19Detection & tolerances14Markets as routers13Prediction-market design9Paying contributors8Decentralised training systems23Digital twins & sim-to-real164LLM chip design loops40Robot manufacturing loops53Ambient AI assistant46Self-replicating manufacturing34Simulation-first design64Engineered clothing45Task-specific AI accelerators40Algorithmic chip-design loops42Open-source purpose-built robots44The software-engineering loop43Adversarial human imitation41Open-source residential construction41Autonomous multi-angle capture34Sensing, imaging and 3D31Energy supply and dense energy storage39Robot societies among people31Agentic co-design of low-cost robots27Laundry-hanging robots25Machine learning on qualitative data23Robots on the building site27Civilizations & group psychology1History & human sciences1
Produced with AI assistance (Claude). Results are from the code in this repository and are simulated or stylised unless the abstract says otherwise; nothing here implies affiliation with or endorsement by any lab, researcher or company cited. MIT licensed.