How sure is the 93% nearest-line equivalence predictor?

What it found

Uncertainty for eda-equivpredict, replayed from the same eda-verify log. The 27 survivors sit on only 11 distinct source lines, so a line-cluster bootstrap gives a 95% interval of 76.5–100% around the observed 25 of 27 (92.6%), against a 59.3% majority-class baseline, and a permutation test that shuffles line groups gives p = 0.012. One design, one solver, predictors chosen with the log in view. Stylised.

A figure from its own output

Paper, code and results

• README and method
• Experiments and results
• Code
• Tests

More in Algorithmic chip-design loops

• Does "the mutant synthesises to the golden netlist" predict formal equivalence on a second design family?Tests whether an identical post-synthesis netlist predicts formal equivalence on a second design (systolic array) as well as the MAC engine: right 3 of 3 times but finds only 3 of 28 proved equivalents (stylised, two designs).
• Do synthesized-netlist features predict which simulation survivors Yosys can prove equivalent?Tests whether Yosys netlist features (cell-count change, miter size) predict which eda-verify survivors are provably equivalent: none beats the 59.3% majority class under leave-one-line-out, against 92.6% for source line (negative, stylised, one design).
• Do fan-in cone features of a mutant predict which simulation survivors Yosys can prove equivalent?Fan-in cone features (changed sinks, cone gates and depth, registers reached) from hashed gate-level netlists vs which eda-verify survivors are provably equivalent: none beats the 59.3% majority class under leave-one-line-out (best 13 of 27; negative, stylised, one design).
• Is the farthest-point spread's lead over random draws, when adapting a chip-design cost model to a new family, bigger than split noise?eda-adaptfixed re-run with 40 splits per family and bootstrap intervals: for the linear model the spread beats random draws at every k (about 1 point of error at 3 to 8 designs, 2.5 at 2), but for ridge it loses at 3 designs and one family (sorter) reverses the sign, so the pooled interval is narrower than the real uncertainty.
• Does the farthest-point spread's lead over random draws, when adapting a chip-design cost model to a new family, survive resampling the designs themselves?eda-adaptci re-run with a design-level bootstrap: the intervals widen by up to 2.5x, the spread's lead holds at 5 and 8 designs for both models (about 0.5 to 1.1 points) and at 2 for the linear model, but the linear k = 3 lead and the ridge k = 2 lead include zero, and the ridge k = 3 loss reported earlier is not established.
• How many placed designs from a new family does an area surrogate need, and does the answer change when the target is placed area instead of a synthesis cell count?The eda-adapt few-shot curves refitted to OpenROAD-placed area on 133 SKY130 designs: an unseen family's placed area is off by 39% (ridge), one placed design cuts it to 23%, about three reach 20%, as for cell counts.

All 42 Algorithmic chip-design loops projects →

Other research topics

Proper scoring & elicitation28Effort & peer prediction9Verification games & slashing19Detection & tolerances14Markets as routers13Prediction-market design9Paying contributors8Decentralised training systems23Digital twins & sim-to-real164LLM chip design loops40Robot manufacturing loops53Ambient AI assistant46Self-replicating manufacturing34Simulation-first design64Engineered clothing45Task-specific AI accelerators40Algorithmic chip-design loops42Open-source purpose-built robots44The software-engineering loop43Adversarial human imitation41Open-source residential construction41Autonomous multi-angle capture34Sensing, imaging and 3D31Energy supply and dense energy storage39Robot societies among people31Agentic co-design of low-cost robots27Laundry-hanging robots25Machine learning on qualitative data23Robots on the building site27Civilizations & group psychology1History & human sciences1

Produced with AI assistance (Claude). Results are from the code in this repository and are simulated or stylised unless the abstract says otherwise; nothing here implies affiliation with or endorsement by any lab, researcher or company cited. MIT licensed.