Does weight-sharing lookup-table inference beat a multiply-accumulate engine for a small accelerator?

research/accel-lut

Weight-sharing product-table inference against int8 MAC engines on the digits MLP: negative at this size, break-even only for wide layers with cheap table reads. Stylised, modelled energy.

6 tests

What it found

A k-means weight-sharing codebook with a per-input product table (no multiplier per weight) against int8 serial MAC engines, on the 64-32-10 digits MLP of accel-sparse, dense and 75% pruned, over 10 splits. Negative: within 2 points of float32 the table engine does not beat int8 (dense: 3.51 nJ for 8 shared values against 3.56 nJ for int8; pruned: 1.55 against 1.51 nJ), and it only looks better against a baseline that rereads activations per weight. An analytic sweep puts the break-even at layers of about 128 outputs or more, and then only if table reads cost a quarter of an array read. Stylised: assumed 45 nm constants, no retraining or RTL, substitute task.

A figure from its own output

Figure produced by research/accel-lut: frontier — accel-lut/experiments/frontier.png, written by the project's own experiment code.

Paper, code and results

More in Task-specific AI accelerators

Does a wider hidden layer make weight-sharing product-table inference beat an int8 MAC engine?Product-table inference on trained wide digits MLPs (32, 128, 512 hidden): at best 14% under int8 energy, only with an assumed cheap table read. Stylised, modelled energy.
Does fine-tuning the shared weight values rescue low-K lookup-table inference?Fine-tuning the shared values of accel-lut's product-table engine: rescues K=2 in the 75%-pruned digits MLP (14% under int8 energy only at an assumed cheap table read), not the dense network. Stylised, modelled energy.
What does a product-table inference engine cost in RTL, against a serial MAC engine?The product-table engine and a serial MAC engine in Verilog, bit-exact against NumPy and synthesised with Yosys: negative, the table needs an accumulator per output (stylised)
Does a tiny always-on keyword model that wakes a bigger one beat the best single model on energy?Classifier-only energy of two-stage keyword-spotting cascades vs 40 single MLPs retrained on the original Speech Commands v0.02 (5 seeds, 200 models, about 41,000 cascades per seed, thresholds chosen on validation): cascades cut modelled energy by 0 to 28% at the same validation target, losing 0.3 to 1.4 test points, and the saving is under 1% of a system whose MFCC front end costs about 50 times the classifier
Does the MFCC front end cost more than the keyword-spotting network it feeds?An instrumented Q15 MFCC front end (512-point FFT, 40 mel bands, DCT) costs 7,532 multiplies per 20 ms frame against the 5,184-MAC accel-kws classifier per inference; under assumed 45 nm constants the front end is 8 to 150 times the classifier's power, and 16-bit FFT scaling decides its accuracy (synthetic frames, no Speech Commands)
Does block floating point fix the 16-bit FFT scaling problem in an always-on MFCC front end?A shared data-chosen exponent for the 16-bit FFT in accel-frontend's MFCC path cuts cepstral error to 0.016 to 0.081 of the spread at -60 to -30 dBFS (one shift per stage: 0.35 to 0.72) for about +0.5% modelled energy, but no better than 18 to 20-bit fixed schedules on loud input (synthetic frames, no Speech Commands)

All 40 Task-specific AI accelerators projects →

Other research topics

Proper scoring & elicitation28 Effort & peer prediction9 Verification games & slashing19 Detection & tolerances14 Markets as routers13 Prediction-market design9 Paying contributors8 Decentralised training systems23 Digital twins & sim-to-real164 LLM chip design loops40 Robot manufacturing loops53 Ambient AI assistant46 Self-replicating manufacturing34 Simulation-first design64 Engineered clothing45 Task-specific AI accelerators40 Algorithmic chip-design loops42 Open-source purpose-built robots44 The software-engineering loop43 Adversarial human imitation41 Open-source residential construction41 Autonomous multi-angle capture34 Sensing, imaging and 3D31 Energy supply and dense energy storage39 Robot societies among people31 Agentic co-design of low-cost robots27 Laundry-hanging robots25 Machine learning on qualitative data23 Robots on the building site27 Civilizations & group psychology1 History & human sciences1

Produced with AI assistance (Claude). Results are from the code in this repository and are simulated or stylised unless the abstract says otherwise; nothing here implies affiliation with or endorsement by any lab, researcher or company cited. MIT licensed.