Written by an AI agent working in this repository.
Earlier milestones of the llm-coding-inference programme found that splitting the human audit of LLM-coded text into the coder's own "yes" and "no" cells makes intervals about 20% narrower when the coder is informative and the category is common (GoEmotions admiration, 8%), but wastes audit items when the coder carries no signal (NaijaSenti, zero-shot Qwen coders). This run tested a simple screen: use a 30% pilot to estimate the coder's informedness J (sensitivity + specificity - 1), use coder cells with a 35% floor if J_hat is at least 0.3, otherwise pool.
Results (500 draws per cell, coverage ±2 points). Where the coder is empty (NaijaSenti 0.5B and 3B, true J 0.01 and 0.06) the screen pooled in every draw and matched the pooled half-width, where always-cells was 22% wider at n = 400. For admiration (J 0.65) it chose cells in 99 to 100% of draws and kept the whole gain (half-width 1.122 vs 1.415 points at n = 1000). It does not fix rare categories: fear (1.2%) and sadness (2.3%) covered 75 and 87% at n = 400 (pooled: about 95%), because a pilot with one or two positives cannot estimate J. The screen was still 25 and 4 points better than always-cells there.
Practical rule: pool unless the category is common and the pilot says the coder is informative. One of four pre-registered predictions was partly refuted (the screen beat always-cells in rare categories, which I had predicted it would not). The informative-coder half still rests on a single category and model.
Code, raw table and the predictions: research/llm-coding-inference/ (LOG.md, milestone 6ak; experiments/h6ak_screen.py).