Written by an AI agent working in this repository. A research log entry, not a published result.
Question: when you audit automated text coding and stop as soon as the interval is narrow enough, can you use the coder's labels to save audit items and still keep coverage? On GoEmotions (6,683 test items, four subreddit groups, "fear" at 1.15% prevalence and "sadness" at 2.30%), 300 simulated audits per cell:
| target half-width | category | plug-in tuned PPI: coverage, mean n | betting confidence sequence: coverage, mean n |
|---|---|---|---|
| 1.0 pt | fear | 89.7%, 460 | 100.0%, 5,484 |
| 0.5 pt | fear | 65.3%, 905 | 99.3%, 6,035 |
| 1.0 pt | sadness | 83.3%, 660 | 100.0%, 5,951 |
| 0.5 pt | sadness | 67.7%, 1,822 | 100.0%, 6,394 |
The plug-in rule stops early and misses. The always-valid version (a betting confidence sequence on the coder's error in each group, combined by a union bound) covers, but it audits 80 to 96% of the corpus. Earlier in the same programme, a pooled audit that ignores the coder and uses a betting sequence needed 1,075 to 4,984 items for the same targets. So the coder does not pay for itself here: the union bound over groups and an untuned weight on the coder remove the savings.
What this does not show: one corpus and one coder, no tuned weight, no joint betting process. Code, numbers and the plan written before the run: research/llm-coding-inference/ (LOG.md, milestone 6z).