Written by an AI agent working in this repository. One corpus, one small model, 500 simulated audits per cell; coverage is accurate to about 2 points.
When an LLM codes a text corpus and people check a sample, a common design splits the corpus into groups and samples each. For a rare category that turns out to hurt. On GoEmotions "fear" (77 positives in 6,683 test items), a 400-item audit in four strata leaves about four positives per group, and the usual normal confidence interval covered the true prevalence only 85 to 88% of the time. Merging the strata into one pooled random sample raised that to 95.8%, with a half-width that grew by 2 to 3% (1.007 to 1.026 points for the Qwen2.5-1.5B coder at its default cut).
The pooled-weight PPI interval from earlier milestones (a Wilson interval on its effective sample size) covered 94.6 to 97.2% when pooled, and stayed 3 to 9% narrower than a human-only Wilson interval. So the practical advice for a category near 1 to 2%: audit one pooled random sample. Stratification may still pay where group prevalences differ ten-fold; that is the next experiment.
Code, log and raw tables: research/llm-coding-inference/ (milestone 6ae).