Does a tiny always-on keyword model that wakes a bigger one beat the best single model on energy?

research/accel-cascade

Classifier-only energy of two-stage keyword-spotting cascades vs 40 single MLPs retrained on the original Speech Commands v0.02 (5 seeds, 200 models, about 41,000 cascades per seed, thresholds chosen on validation): cascades cut modelled energy by 0 to 28% at the same validation target, losing 0.3 to 1.4 test points, and the saving is under 1% of a system whose MFCC front end costs about 50 times the classifier

6 tests

What it found

Track 8 asked whether co-design beats pairing the best model with a generic engine; this project asks a runtime version: does a small model that answers confident clips and escalates unsure ones to a bigger model beat the best single model? It retrains the accel-kws grid (segments 7/12, hidden 8 to 128, weights 2/3/4/8 bits; 40 integer models) on the original Speech Commands v0.02 tarball with its own speaker-disjoint splits (33,895 / 4,084 / 4,509 clips), 5 seeds. Per seed, every ordered small/big pair is a cascade (about 41,000 per seed), escalating when the small model's integer logit margin is below a threshold chosen on validation only. Energy is the classifier's modelled MAC energy (constants checked against Horowitz 2014 in accel-horowitzcheck) plus the escalated fraction of the big model. At validation targets of 0.65 to 0.74 the cheapest cascade uses 0.73 to 0.98 of the cheapest single model's energy (mean ratios 0.82, 0.98, 0.73, 0.77 at 0.65, 0.70, 0.72, 0.74; 0.92 and 0.98 at 0.55 and 0.60 where only some seeds benefit), but its test accuracy is 0.3 to 1.4 points lower (test-set standard error about 0.7 points). The best single model reaches 0.755 test. Limits: classifier energy only; accel-frontend found the MFCC front end draws about 50 times the classifier, which swamps any cascade saving; clips are isolated words, not a stream dominated by silence; modelled, not measured, energy; one hidden layer; unknown class subsampled.

Paper, code and results

More in Task-specific AI accelerators

Does the MFCC front end cost more than the keyword-spotting network it feeds?An instrumented Q15 MFCC front end (512-point FFT, 40 mel bands, DCT) costs 7,532 multiplies per 20 ms frame against the 5,184-MAC accel-kws classifier per inference; under assumed 45 nm constants the front end is 8 to 150 times the classifier's power, and 16-bit FFT scaling decides its accuracy (synthetic frames, no Speech Commands)
Does block floating point fix the 16-bit FFT scaling problem in an always-on MFCC front end?A shared data-chosen exponent for the 16-bit FFT in accel-frontend's MFCC path cuts cepstral error to 0.016 to 0.081 of the spread at -60 to -30 dBFS (one shift per stage: 0.35 to 0.72) for about +0.5% modelled energy, but no better than 18 to 20-bit fixed schedules on loud input (synthetic frames, no Speech Commands)
Do the accel-* energy constants match Horowitz (2014), and does fixing the array-size term change accel-kws's conclusions?All 34 energy constants in the accel-* projects that have a counterpart in Horowitz's Fig. 1.1.9 are within 4% of it; re-costing accel-kws with the figure's array-size law (energy about the square root of size) moves energy to 0.23 to 0.81 of as built but keeps 17 of 18 front designs (Spearman 0.991)
Does weight-sharing lookup-table inference beat an int8 MAC engine once small arrays cost less to read?Pricing every array by its own size flips accel-lut's negative result once read energy grows faster than size^0.26 (dense) or size^0.28 (pruned); Horowitz's slope is 0.5 above 8 KB but there is no point below it, so the verdict is conditional
Is the OpenRAM access energy in SKY130 liberty files an energy at all, and does accel-pdkprice's weight-stationary verdict survive?OpenRAM's clk0 figure is an average power in mW over one cycle of an unstated period, not pJ: the unpacked verdict (register-file WS wins) holds for any period above 3 ns, while the packed verdict flips between 1 and 31 ns, so accel-pdkprice's packed conclusion is conditional
Once the MFCC front end is counted, does the energy-optimal keyword model change?Joining accel-cascade's committed results with accel-frontend's 46.8 nJ per frame: cascades' classifier-only saving of up to 28% becomes at most 0.42% of whole-system energy, while the most accurate model (about 0.75 test accuracy) costs 23 to 26 times the cheapest classifier but only 5% more system energy (modelled, not measured)

All 40 Task-specific AI accelerators projects →

Other research topics

Proper scoring & elicitation28 Effort & peer prediction9 Verification games & slashing19 Detection & tolerances14 Markets as routers13 Prediction-market design9 Paying contributors8 Decentralised training systems23 Digital twins & sim-to-real164 LLM chip design loops40 Robot manufacturing loops53 Ambient AI assistant46 Self-replicating manufacturing34 Simulation-first design64 Engineered clothing45 Task-specific AI accelerators40 Algorithmic chip-design loops42 Open-source purpose-built robots44 The software-engineering loop43 Adversarial human imitation41 Open-source residential construction41 Autonomous multi-angle capture34 Sensing, imaging and 3D31 Energy supply and dense energy storage39 Robot societies among people31 Agentic co-design of low-cost robots27 Laundry-hanging robots25 Machine learning on qualitative data23 Robots on the building site27 Civilizations & group psychology1 History & human sciences1

Produced with AI assistance (Claude). Results are from the code in this repository and are simulated or stylised unless the abstract says otherwise; nothing here implies affiliation with or endorsement by any lab, researcher or company cited. MIT licensed.