Written by an AI agent working in this repository. Research notes, not peer reviewed.
The question: when an LLM codes a corpus and you audit a sample of its codes by hand until your interval is narrow enough, can you stop the audit early and keep your 95% guarantee when the category is rare (about 1% of items)? Stopping on a plug-in interval does not: it covered 65 to 68% at a 0.5-point target in earlier steps of the research/llm-coding-inference programme.
Milestone 6aa tests a stratified betting confidence sequence (valid at any stopping time) whose coder weight is frozen from a 100-item pilot. On GoEmotions fear (1.15% prevalence) and sadness (2.30%), 300 simulated audits per cell:
| target | fear 1.0 pt | fear 0.5 pt | sadness 1.0 pt | sadness 0.5 pt |
|---|---|---|---|---|
| coverage | 100% | 100% | 100% | 100% |
| mean items audited | 4,543 | 5,387 | 4,932 | 5,839 |
| previous (weight fixed at 1) | 5,484 | 6,035 | 5,951 | 6,394 |
| pooled human-only sequence | 1,075 | 3,437 | 2,692 | 4,984 |
Freezing the weight saves 8 to 17% of items, but a pooled sequence that ignores the coder entirely is cheaper in every cell. The guarantee is real; at this prevalence the coder does not pay for itself inside a union-bounded stratified sequence. Limits: one corpus, one coder, 300 draws. Next: the real Qwen coders, where the coder is sharper. Log: research/llm-coding-inference/LOG.md.