Can cheap features say which simulation survivors formal can prove equivalent?

What it found

Replay of the eda-verify log on the 27 mutants that 16 directed vectors do not kill (16 provably equivalent, 9 timeouts, 2 formal kills). Predicting from the mutated source line and running formal only on predicted-equivalent survivors costs 3,298 CPU-s against 7,414 s for always running formal (−55%) and still finds 14 of 16 proofs, 93% accurate under leave-one-line-out; smoother rules (3-NN, a one-split rule) find 12 of 16 and operator kind is no better than guessing. The signal is the design region the solver finds hard, not a transferable rule. Stylised: 27 mutants, one design, one solver; nothing new was simulated.

A figure from its own output

Paper, code and results

• README and method
• Experiments and results
• Code
• Tests

More in Algorithmic chip-design loops

• How sure is the 93% nearest-line equivalence predictor?Puts a line-clustered interval on eda-equivpredict's 93%: 25 of 27 survivors right, 95% CI 76.5–100% over 11 distinct lines, permutation p = 0.012 (stylised, one design).
• Does "the mutant synthesises to the golden netlist" predict formal equivalence on a second design family?Tests whether an identical post-synthesis netlist predicts formal equivalence on a second design (systolic array) as well as the MAC engine: right 3 of 3 times but finds only 3 of 28 proved equivalents (stylised, two designs).
• Do synthesized-netlist features predict which simulation survivors Yosys can prove equivalent?Tests whether Yosys netlist features (cell-count change, miter size) predict which eda-verify survivors are provably equivalent: none beats the 59.3% majority class under leave-one-line-out, against 92.6% for source line (negative, stylised, one design).
• Do fan-in cone features of a mutant predict which simulation survivors Yosys can prove equivalent?Fan-in cone features (changed sinks, cone gates and depth, registers reached) from hashed gate-level netlists vs which eda-verify survivors are provably equivalent: none beats the 59.3% majority class under leave-one-line-out (best 13 of 27; negative, stylised, one design).
• Is the farthest-point spread's lead over random draws, when adapting a chip-design cost model to a new family, bigger than split noise?eda-adaptfixed re-run with 40 splits per family and bootstrap intervals: for the linear model the spread beats random draws at every k (about 1 point of error at 3 to 8 designs, 2.5 at 2), but for ridge it loses at 3 designs and one family (sorter) reverses the sign, so the pooled interval is narrower than the real uncertainty.
• Does the farthest-point spread's lead over random draws, when adapting a chip-design cost model to a new family, survive resampling the designs themselves?eda-adaptci re-run with a design-level bootstrap: the intervals widen by up to 2.5x, the spread's lead holds at 5 and 8 designs for both models (about 0.5 to 1.1 points) and at 2 for the linear model, but the linear k = 3 lead and the ridge k = 2 lead include zero, and the ridge k = 3 loss reported earlier is not established.

All 42 Algorithmic chip-design loops projects →

Other research topics

Proper scoring & elicitation28Effort & peer prediction9Verification games & slashing19Detection & tolerances14Markets as routers13Prediction-market design9Paying contributors8Decentralised training systems23Digital twins & sim-to-real164LLM chip design loops40Robot manufacturing loops53Ambient AI assistant46Self-replicating manufacturing34Simulation-first design64Engineered clothing45Task-specific AI accelerators40Algorithmic chip-design loops42Open-source purpose-built robots44The software-engineering loop43Adversarial human imitation41Open-source residential construction41Autonomous multi-angle capture34Sensing, imaging and 3D31Energy supply and dense energy storage39Robot societies among people31Agentic co-design of low-cost robots27Laundry-hanging robots25Machine learning on qualitative data23Robots on the building site27Civilizations & group psychology1History & human sciences1

Produced with AI assistance (Claude). Results are from the code in this repository and are simulated or stylised unless the abstract says otherwise; nothing here implies affiliation with or endorsement by any lab, researcher or company cited. MIT licensed.