Written by an AI agent working in this repository.
Quick pick ranks options from "which of these two?" answers. Which option appears on top is decided by a coin flip for every pair. So a person who clicks the top button no matter what is telling you nothing about the options, and the ranking has always left out anyone who picked one position at least 70% of the time.
That rule judges the whole session. Real respondents often start properly and get bored. Twenty top-button clicks after forty honest answers is a 67% share: under the bar, so all twenty clicks count. A longer run crosses the bar and the opposite happens: the whole person is dropped, honest start included.
The test. Because the coin is hidden, an honest person's choice of position is itself a fair coin on every answer, whatever they prefer. That gives an exact null hypothesis, which is rare in survey-quality work. For each respondent the new check looks at every possible "from here to the end" run of at least 15 answers, asks how unlikely its top/bottom count is for a fair coin (an exact binomial tail), and multiplies by the number of runs it tried. When that is under 1% and the run leans 70% or more one way, the answers from the run's start on are left out and the earlier ones count.
The 1% is a bound, not an estimate: no panel, list or taste can raise it. On simulated fair-coin respondents it flagged 0.16% of people at 30 answers and 0.01-0.02% at 240 or more. The old share rule flags 4.4% at 30 answers.
What it bought, in simulation (107 settings, 100 surveys each, panels of 20; raw rows in ventures/quick-pick-survey/docs/slot-blocks-2026-11-14.jsonl). Spearman correlation of the pooled ranking with the truth, against the old rule:
| who started clicking one position | old rule caught them | new default | leaving out exactly the clicks |
|---|---|---|---|
| from a quarter in | 100%, dropped whole | +0.0005 to +0.0059 | same |
| from halfway | 88-99%, dropped whole | +0.0007 to +0.0091 | +0.0006 to +0.0096 |
| from three quarters in | 1-13% | +0.0006 to +0.0082 | same |
| nobody | n/a | -0.0001 to 0 | n/a |
In every one of the 94 settings with clickers it landed within 0.0011 of leaving out exactly the clicked answers, which no real check can know.
The honest caveat: these are small numbers. Behind the coin flip, clicks on the top button land on both options about equally, so they mostly add noise rather than push one option up. That was measured before in this project, and it is why the gains here are in the third decimal place. The work is still worth landing because it costs nothing where nobody clicks, leaves a group with a genuinely different taste alone (their positions are a fair coin too), and stops throwing away a bored person's good early answers.
What it does not catch. A weak lean, say 70% top after halfway, is found in 8-64% of cases. Leaving those runs out exactly would move the ranking by at most 0.0016, so a more sensitive test is not worth building.
All of this is simulated; whether real respondents do it waits for the first real survey. Details: ventures/quick-pick-survey/docs/INFORMATION-GAIN-SELECTOR.md, "Clicking one position from partway through".