How much does noisy scoring cost the secretary problem, and should the 1/e cutoff move?

What it found

Simulation of the classical cutoff rule for hiring the best of n = 50 candidates when only a noisy score is seen (noise sd 0 to 4 times the quality sd, 40,000 trials per cell, checked against the exact noiseless formula). The best cutoff barely moves, 19 with no noise and 16–17 at every noise level; the chance of picking the true best falls from 0.373 to 0.145 at noise 1 and 0.043 at noise 4 (lottery: 0.02). Keeping the classical cutoff costs at most 0.006. A high mean true rank wants a much earlier cutoff (7–9). Cutoff rules only, Gaussian, one setting; the optimal noisy policy is not computed.

A figure from its own output

Paper, code and results

• README and method
• Experiments and results
• Tests

More in Digital twins & sim-to-real

• How many simulator rollouts does a sequential test save when certifying a failure rate?How many simulator rollouts a sequential test saves when certifying a failure rate: exact dynamic programming shows Wald's SPRT uses about half the rollouts of the best fixed test at either hypothesis (79/113 vs 181 for 1% vs 5%), and 6–17% fewer at its worst point in between.
• How long can a sequential rollout test run, and what does capping it cost?How long can a sequential rollout test run, and what does capping it cost? Exact: the SPRT's 99th percentile is 309 rollouts against a 79 mean (1% vs 5%; fixed test 181), 28% of runs exceed the fixed size at the midpoint rate; capping at the fixed size adds 34–38% to the error, a cap of 1.5 times it keeps the error at 0.053 to 0.057 for a mean within 2% of uncapped.
• With noisy scores, does a "beat the record by a margin" rule pick the best candidate more often than the best cutoff?With noisy scores, a rule that must beat the earlier best by a margin raises P(pick the true best) from 0.233 to 0.244 at noise 0.5 and 0.143 to 0.149 at noise 1 (n = 50, fresh-trial test); no gain at noise 2; simulation.
• Does a sampled-data loop tuned on a fixed-period twin survive random sampling intervals?Does a sampled-data loop tuned on a fixed-period twin survive random sampling intervals? Uniform jitter up to ±100% never destabilises it, but missed samples do: almost-surely stable up to miss probability 0.556, mean-square stable only to 0.270 (0.05 for a deadbeat tuning); at 0.4 all 2000 simulated runs converge. Stylised scalar model.
• Does retuning the gain on the sampling-interval law, instead of the fixed-period twin, remove the missed-sample penalty?Does retuning the gain on the sampling-interval law, instead of the fixed-period twin, remove the missed-sample penalty? Yes for this plant: the mean-square-optimal gain is stable at every miss probability (worst E g² 0.478 vs a twin-gain limit of 0.270), at the price of a slower nominal loop; a gain designed for q = 0.3 stays stable at every true q. Stylised scalar model.
• What does a 1:N model of a cargo UAV tell you about the 1:1 aircraft?Research programme (proposal stage): per-quantity error budget for a Froude-scaled cargo aircraft model, with a stdlib scaling calculator; first-order result: at 1:10 the model needs 17 to 35% more power than Froude scaling predicts

All 164 Digital twins & sim-to-real projects →

Other research topics

Proper scoring & elicitation28Effort & peer prediction9Verification games & slashing19Detection & tolerances14Markets as routers13Prediction-market design9Paying contributors8Decentralised training systems23Digital twins & sim-to-real164LLM chip design loops40Robot manufacturing loops53Ambient AI assistant46Self-replicating manufacturing34Simulation-first design64Engineered clothing45Task-specific AI accelerators40Algorithmic chip-design loops42Open-source purpose-built robots44The software-engineering loop43Adversarial human imitation41Open-source residential construction41Autonomous multi-angle capture34Sensing, imaging and 3D31Energy supply and dense energy storage39Robot societies among people31Agentic co-design of low-cost robots27Laundry-hanging robots25Machine learning on qualitative data23Robots on the building site27Civilizations & group psychology1History & human sciences1

Produced with AI assistance (Claude). Results are from the code in this repository and are simulated or stylised unless the abstract says otherwise; nothing here implies affiliation with or endorsement by any lab, researcher or company cited. MIT licensed.