Quick Pick / weigh criteria
A weighted decision matrix you fill in by picking pairs
Choosing a flat, a vendor or a laptop means trading off price against everything else, and a spreadsheet asks you to type weights like "price 30%" that nobody really knows. Here you only answer two kinds of easy question: which criterion matters more, and which option is better on one criterion. The page works out the weights and the scores, and shows how sure it is.
1. Options and criteria
Keys: ← / → a bit better, Shift+← / Shift+→ much better (or 1 2 3 4 from left to right), ↓ about equal, U undo. "Much better" tells the page more than a plain pick, so it settles sooner; use it when the gap is clear to you, not out of habit.
Your decision
| # | Option | Rank range | Best | Strongest on |
|---|
How much each criterion counted
| Criterion | Weight | Range | Would change the winner |
|---|
A criterion's weight comes from how often you said it matters more than the others: if you pick Rent over Space 3 times in 4, Rent counts about 3 times as much. The ranges are honest about how little a handful of answers pins down; a wide weight range is the usual reason a rank range stays wide. Would change the winner: the weight this criterion would need (the others keeping their proportions) for another option to come out best. If that weight sits inside the range, the decision really does turn on how much you care about it.
How it works
- Two models in one. "Which matters more?" answers fit a Bradley-Terry model over the criteria; a criterion's weight is its share, so the weights add to 100%. "Which is better on Commute?" answers fit a Bradley-Terry model over the options for that criterion. An option's overall score is the weighted sum of its scores. It is the same refit the list ranker and the Quick Pick survey app run.
- The next question is whichever one, of either kind, is expected to do most to settle the overall order: it targets neighbouring options that could still swap, and asks about the weight or the criterion that decides between them. Questions about a criterion that cannot change the order are skipped. Sides are random, so screen position cannot creep in.
- Ranges include doubt about the weights. The page draws thousands of plausible weightings and scores consistent with your answers; the rank range is where 90% of them put each option, and "Best" is how often it came first.
- When it stops: as soon as there is under a 10% chance that any option beats the leader by more than a practical tie (a weighted edge of about 56 to 44 in a head-to-head pick), or when the whole order is 90% settled, or at a cap. You came to choose one option, so it does not wait for the order below the winner to settle; with only the order rule, nearly every simulated run hit the cap. With "much" answers it checks once more before stopping, reading your "much" as cautiously as it can, so someone who says "much" freely is not taken as surer than they are. If you keep going past the point where it shows results, every extra answer is another look at the results and another chance to stop on a lucky run of picks, so it asks for a little more certainty the longer it goes (about 93% at twice that point, 94% at three times). You can stop at any time. Measured on simulated people: below.
- "Much" or "a bit". Each pick says how big the gap feels. The page reads that with an ordinal model: a "much" answer means the gap felt past your own threshold, and that threshold is estimated from your answers, so someone who says "much" for nearly everything has each "much" count for less. On simulated people the graded answers named the true best more often and stopped sooner than plain picks (measured).
- Why not type the weights? People are poor at stating weights directly but consistent at comparing two things. This is the idea behind AHP (the analytic hierarchy process), without its full table of every pair: the picker asks only the comparisons that matter.
Measured
Simulated people with random weights and scores, answering with ordinary noise, are in docs/DECIDE-PAGE.md, with the script that produced them. Graded answers ("much" or "a bit"), on 300 simulated people: the true best came first 182 to 203 times against 165 with plain picks, at about the same number of picks, and the stop kept its 90% promise for people who use "much" sparingly, normally or inconsistently.
One list and one question? The list ranker is simpler. Running a survey with many respondents? The full app is open source: deploy your own copy.
Privacy and usage counting
Everything runs in your browser. Your options, criteria and answers are never uploaded; we have no accounts, tracking cookies or analytics vendor. Progress is kept in this browser's local storage so a reload does not lose it. The page counts anonymous events in local storage (decisions started, picks made, results shown, codes copied, team combines), nothing about who you are or what you compared, and the site counts one page view per load without any identifier.