Does a larger batch let a systolic array catch a MAC-lane engine on a tiny quantised MLP?
research/chip-systolic-batch
Batch-size sweep of the systolic MLP engine (R = 4, 8, 16): the array's cells-times-cycles disadvantage against chip-accel's MAC lanes widens from 1.6x to 2.9x because its area grows with the batch (negative, stylised, generic-gate counts).
What it found
A batch-size sweep (R = 4, 8, 16, 4-bit weights) of the output-stationary systolic generator from chip-systolic, simulated in Icarus Verilog and matched to an integer NumPy reference and the cycle formula in all 6 configurations, then scored with Yosys against the MAC-lane engine of chip-accel. A negative result: the best array has 1.6×, 2.1× and 2.9× the cells×cycles of 4 MAC lanes at R = 4, 8 and 16, because array area grows with the batch while MAC-lane area does not, even though it needs far fewer cycles (83 against 464 at R = 16). Stylised: fan-in 8, one network, an array that grows a row per vector, gate counts rather than area, no clock, power or layout.
A figure from its own output
Paper, code and results
More in LLM chip design loops
- Does a small systolic array reused over a stream of vectors beat a MAC-lane engine on a tiny quantised MLP?Small reused systolic arrays (R = 1, 2, 4 rows) against
chip-accel's MAC lanes: even the best, 1x4, has 1.35x the cells-times-cycles per vector (65702 vs 48546) at fan-in 8 (negative, stylised, generic-gate counts). - Does a wider fan-in let a small systolic array catch a MAC-lane engine on a tiny quantised MLP?Fan-in sweep (F = 8, 16, 32) of small systolic arrays against
chip-accel's MAC lanes: the best array costs 1.36x, 1.25x and 1.26x the cells-times-cycles per vector of the best MAC engine, so the gap narrows but does not close (negative, stylised, generic-gate counts). - Do the optima of the fan-in sweep move when columns and MAC lanes go past 8?Extends the fan-in sweep past its edges (C = 16, P = 32): the optima move inward and the best array still costs 1.25x (F = 16) and 1.20x (F = 32) the cells-times-cycles per vector of the best MAC engine (negative, stylised, generic-gate counts).
- Does the array/MAC-engine gap change when the hidden width is fixed at 8 and only the input fan-in grows?Fixes the hidden width at 8 and sweeps the input fan-in F = 8, 16, 32: the best MAC engine still beats the best systolic array, by 1.35x, 1.22x and 1.20x in cells-times-cycles per vector (negative, stylised, generic-gate counts).
- Which MAC-lane count is best for a tiny quantised MLP if every P from 1 to 20 is built and checked?Builds and RTL-checks every MAC-lane count P = 1 to 20 on the F-16-4 network (F = 16, 32): the best P is 8 and 16, no non-power-of-two wins, and P above 16 only adds cells (stylised, generic-gate counts).
- Does the MAC-engine lead survive a wider hidden layer (H = 16) and more MAC lanes?Widens the hidden layer to 16 (F = 16, 32) and lets the MAC engine reach P = 16: the best MAC engine still beats the best systolic array, by 1.30x and 1.27x in cells-times-cycles per vector, and at F = 32 its best lane count is again the largest tried (negative, stylised, generic-gate counts).
All 40 LLM chip design loops projects →
Other research topics
Proper scoring & elicitation28Effort & peer prediction9Verification games & slashing19Detection & tolerances14Markets as routers13Prediction-market design9Paying contributors8Decentralised training systems23Digital twins & sim-to-real164LLM chip design loops40Robot manufacturing loops53Ambient AI assistant46Self-replicating manufacturing34Simulation-first design64Engineered clothing45Task-specific AI accelerators40Algorithmic chip-design loops42Open-source purpose-built robots44The software-engineering loop43Adversarial human imitation41Open-source residential construction41Autonomous multi-angle capture34Sensing, imaging and 3D31Energy supply and dense energy storage39Robot societies among people31Agentic co-design of low-cost robots27Laundry-hanging robots25Machine learning on qualitative data23Robots on the building site27Civilizations & group psychology1History & human sciences1
Produced with AI assistance (Claude). Results are from the code in this repository and are simulated or stylised unless the abstract says otherwise; nothing here implies affiliation with or endorsement by any lab, researcher or company cited. MIT licensed.