Quick Pick / rank a list / how many comparisons

How many pairwise comparisons does it take to rank a list?

Asking every pair once grows with the square of the list: 190 questions for 20 options, 1225 for 50. An adaptive ranker that picks the most informative pair each time needed far fewer. We ran the free ranker's own code on simulated people to see how many.

Rank your own list →

Picks before the ranker stopped

Mean over 24 simulated people per cell (fewest to most in brackets). It stops when the order is 90% settled, or at its cap of about 2 n log2 n.

OptionsEvery pair oncePerfect-sort minimumClear-cutOrdinaryClose callsCap
46511 (11–12)13 (10–16)16 (13–16)16
510715 (14–17)21 (15–24)24 (20–24)24
6151018 (17–21)25 (18–32)30 (20–32)32
8281626 (24–27)40 (29–48)44 (32–48)48
10452233 (31–34)48 (34–67)59 (37–67)67
12662940 (38–42)58 (37–87)78 (53–87)87
151054151 (50–53)73 (52–118)104 (61–118)118
201906272 (69–75)93 (76–152)141 (74–173)173
253008492 (88–95)118 (95–202)189 (96–233)233
30435108112 (106–118)146 (117–222)212 (116–295)295
40780160150 (145–155)204 (154–339)344 (158–426)426
501225215189 (181–195)263 (194–420)398 (193–565)565

Clear-cut: someone who never picks against their own order; Ordinary: someone who picks their favourite over their least favourite 998 times in 1,000, but finds neighbours on a 15-option list a 61/39 call; Close calls: someone who picks their favourite over their least favourite 95 times in 100, and finds neighbours on a 15-option list a 55/45 call. "Perfect-sort minimum" is log2(n!) rounded up, the fewest questions any method could use on someone who never wavers.

What the numbers say

How good was the order at the stop?

Rank correlation (Spearman, 1 = exactly right) between the order shown and the simulated person's real order, and how often the top option shown was really their favourite. "Every pair once" is the same person asked each pair exactly once and refit the same way (n(n-1)/2 questions). The last column is the share of close-call runs that stopped at the cap.

OptionsClear-cut: Spearmantop rightOrdinary: Spearmantop rightClose calls: Spearmantop rightEvery pair once: ordinaryEvery pair once: close callsClose calls at cap
41.00100%0.9796%0.96100%0.900.7979%
51.00100%0.9892%0.9088%0.880.7688%
61.00100%0.96100%0.9067%0.880.7567%
81.00100%0.9683%0.8867%0.910.7658%
101.00100%0.9567%0.8858%0.920.8146%
121.00100%0.9463%0.9050%0.920.8446%
151.00100%0.9458%0.8758%0.940.8558%
201.00100%0.9442%0.8638%0.950.8850%
250.99100%0.9342%0.8525%0.960.8950%
300.99100%0.9333%0.8333%0.960.9129%
400.9992%0.9238%0.8821%0.970.9450%
500.9979%0.9242%0.844%0.980.9438%

The top option is often wrong for close-call and long lists even when the whole order is good: the ranker is built to settle the order, not the winner. If you only need the winner, its "be sure of the favourite" mode spends questions on the front-runners instead.

Only need a shortlist? How many picks for a top 3 or 5

Often the job is "which 3 go on the shortlist", not the whole order. The ranker's "Find my top k" mode asks about the options that could still fall either side of the cut and stops when the shortlist is 90% settled. Below, the same simulated people (16 per cell) ranked the full order with its first k read off, and then ran the shortlist mode. "Right" counts the runs whose shown top k was exactly the person's true top k, as a set.

OptionsPersonFull order: picksrightShortlist mode: picksrightat cap
10, top 3Clear-cut3316/164116/160/16
Ordinary5114/165213/167/16
Close calls584/166110/1612/16
15, top 3Clear-cut5116/166316/160/16
Ordinary734/161009/168/16
Close calls1010/161186/1615/16
20, top 3Clear-cut7116/168216/160/16
Ordinary864/1616112/1611/16
Close calls1405/161735/1616/16
30, top 3Clear-cut11116/1611616/160/16
Ordinary1696/162687/1610/16
Close calls2120/162951/1616/16
10, top 5Clear-cut3316/163716/160/16
Ordinary5112/164714/162/16
Close calls584/16562/169/16
15, top 5Clear-cut5116/166816/160/16
Ordinary737/1610211/167/16
Close calls1014/161156/1613/16
20, top 5Clear-cut7116/169316/160/16
Ordinary860/161467/166/16
Close calls1403/161694/1615/16
30, top 5Clear-cut11111/1613616/160/16
Ordinary1694/162628/169/16
Close calls2120/162951/1616/16

Script: scripts/top-k-sweep.mjs; rows: docs/top-k-2026-11-14.jsonl (arms order and boundary); people as above. A shortlist link carries its k, so everyone you send it to answers for the same cut.

Questions people ask

How many pairwise comparisons do you need to rank n items?
Asking every pair once takes n(n-1)/2: 190 for 20 items, 1225 for 50. An adaptive picker that asks only the informative pairs needed about 93 picks for 20 items and 263 for 50 in our simulations of a person with ordinary consistency (an order nearly as good as every pair once gave: 0.94 against 0.95 Spearman at 20), and 72 and 189 for a person who never contradicts themselves.
Is asking every pair ever the better choice?
On short lists it is fewer questions. Below 12 items, a person with ordinary consistency was asked more questions by the adaptive ranker than there are pairs, because it re-asks close pairs to be sure of them. Compared on the same simulated people, the ranker's extra questions buy a better order (0.07 Spearman on average), so they are worth answering if you have the minute. Above that, every pair once quickly becomes too many questions to ask anyone.
How many comparisons does it take to find the top 3 of 20?
If you only need a shortlist, rank for the cut, not the whole order. For 20 options and a top 3, a simulated person with ordinary consistency got the exact top 3 right 12 times in 16 after about 161 picks with a picker that asks about the options near the cut, against 4 in 16 after 86 picks when the full order was ranked and its first 3 read off. Someone who never contradicts themselves needed about 82, and every pair once would be 190.
What is the fewest comparisons any method could use?
A perfect sort needs at least log2(n!) comparisons, about n log2 n: 62 for 20 items. That bound is for getting the order exactly right from someone who never answers inconsistently. At 30 options the minimum is 108 and a never-wrong person was asked 112: the ranker allows for the odd slip, so it won't call an order settled on one answer per pair. From 40 options it stopped below the minimum (150 against 160; 189 against 215 at 50), because it stops when the order is 90% settled, not exact: those orders scored 0.99 Spearman, nearly but not always perfect.

How this was measured

Each simulated person has true scores evenly spaced over a range (100 logits for clear-cut, 6 for ordinary, 3 for close calls), shuffled, and picks A over B with probability 1/(1+e−(a−b)), never "about equal". The questions come from the ranker's own picker (the pair with the most expected information, sides by coin flip) and it stops by the ranker's own rule: a Bradley-Terry refit after every answer, the order's posterior confidence at 0.9, rechecked at a prior fitted to the person's own answers. Real people may differ: they tire, warm up and have position habits, which these simulations leave out. Script: scripts/picks-needed.mjs; rows: docs/picks-needed-2026-11-14.jsonl; the method and its other measurements.

Ready lists to start from: a feature backlog, your personal values, a name shortlist, a group trip, team goals, customer pain points, a to-do list, or all ready lists.