Is the farthest-point spread's lead over random draws, when adapting a chip-design cost model to a new family, bigger than split noise?

What it found

Same 133 Yosys-synthesised designs in four families, fixed 40% test set per family, new-family weight 5. Over 40 splits × 4 families the paired gap APE(spread) − APE(random) has a 95% bootstrap interval excluding zero for the linear model at k = 2, 3, 5, 8 (median −2.45, −0.71, −1.01, −0.73 points) and for ridge except k = 3, where the spread is worse (+1.00, interval +0.19 to +1.90). The extremes chooser is worse than random at k = 2 to 3. The alu family drives the largest gaps and sorter reverses the sign at k = 3 and 5. The intervals reflect split noise on the same designs, not a population of families, and are narrow for that reason. Inputs are the committed dataset (Yosys 0.33 counts); weights and scaling are untuned defaults.

A figure from its own output

Paper, code and results

• README and method
• Experiments and results
• Code
• Tests

More in Algorithmic chip-design loops

• Does the farthest-point spread's lead over random draws, when adapting a chip-design cost model to a new family, survive resampling the designs themselves?eda-adaptci re-run with a design-level bootstrap: the intervals widen by up to 2.5x, the spread's lead holds at 5 and 8 designs for both models (about 0.5 to 1.1 points) and at 2 for the linear model, but the linear k = 3 lead and the ridge k = 2 lead include zero, and the ridge k = 3 loss reported earlier is not established.
• How many placed designs from a new family does an area surrogate need, and does the answer change when the target is placed area instead of a synthesis cell count?The eda-adapt few-shot curves refitted to OpenROAD-placed area on 133 SKY130 designs: an unseen family's placed area is off by 39% (ridge), one placed design cuts it to 23%, about three reach 20%, as for cell counts.
• Do input-biased random vectors (extreme, sparse, one-hot) catch RTL bugs that uniform random vectors never did?Four biased generators (20,000 vectors per target) catch none of the 26 eda-deepdraw survivors, so none contradicts a formal equivalence proof, and the del@122 mutant is confirmed as truncated output (90 rows for 100 vectors).
• Do the SKY130 area and latency conclusions for a MAC-engine family survive a target clock period?The 40 MAC-engine designs mapped to SKY130 at six shared clock targets: cell count still ranks area (Spearman 0.995+), logic depth ranks delay at only 0.34 to 0.82, and with one shared clock the Pareto front equals the generic one, so the earlier front change came from per-design clocks.
• Do 20,000 random vectors catch any RTL bug that 64 missed, and do they contradict the formal equivalence proofs?20,000 further random vectors catch none of the 25 eda-verify survivors, so none contradicts a formal equivalence proof; a random-visible bug among the 9 formal timeouts has per-vector detection probability below 1.4e-4.
• Can a library-free delay estimate rank accelerator designs by SKY130 critical-path delay better than logic depth?Library-free delay proxies on the generic gate netlist of the 40 MAC engines: gate-type weights leave depth's Spearman 0.611 against SKY130-mapped delay unchanged (0.609), adding fan-out lifts it to 0.83 to 0.90, but gate count alone reaches 0.911, and no proxy recovers the true (area, latency) front.

All 42 Algorithmic chip-design loops projects →

Other research topics

Proper scoring & elicitation28Effort & peer prediction9Verification games & slashing19Detection & tolerances14Markets as routers13Prediction-market design9Paying contributors8Decentralised training systems23Digital twins & sim-to-real164LLM chip design loops40Robot manufacturing loops53Ambient AI assistant46Self-replicating manufacturing34Simulation-first design64Engineered clothing45Task-specific AI accelerators40Algorithmic chip-design loops42Open-source purpose-built robots44The software-engineering loop43Adversarial human imitation41Open-source residential construction41Autonomous multi-angle capture34Sensing, imaging and 3D31Energy supply and dense energy storage39Robot societies among people31Agentic co-design of low-cost robots27Laundry-hanging robots25Machine learning on qualitative data23Robots on the building site27Civilizations & group psychology1History & human sciences1

Produced with AI assistance (Claude). Results are from the code in this repository and are simulated or stylised unless the abstract says otherwise; nothing here implies affiliation with or endorsement by any lab, researcher or company cited. MIT licensed.