Does the MFCC front end cost more than the keyword-spotting network it feeds?

What it found

accel-kws claimed the microphone and MFCC front end, not the MAC array, would set the always-on budget but never counted the front end. This project writes a Q15 integer front end (pre-emphasis, Hamming window, 512-point real FFT as a 256-point complex radix-2 FFT, power, the 247 non-zero mel weights of the real filterbank, a table log, a 10-coefficient DCT) with a counter on every multiply, add and 16-bit word moved. One 20 ms frame takes 7,532 multiplies, 11,009 adds and 15,620 words moved, against 5,184 MACs for the 70-64-11 classifier per inference. Streaming (each frame computed once, 50 per second) the front end draws about 54 times the classifier's modelled power at the assumed constants (8 times counting arithmetic only, 23 to 146 times with memory energy a third to triple), and 211 times if all 49 frames are recomputed for each of 4 inferences per second. Decimating by 2 to halve the FFT saves 115,200 multiplies per second, so a filter costing more than 14.4 MACs per output sample loses. The integer path matches a float reference on a noisy tone (within 0.05 with 2 FFT shifts), but with the usual one right shift per stage and a −61 dBFS microphone floor the cepstral error is 0.19 of their spread at −20 dBFS and 0.68 at −50 dBFS; fewer shifts cut it to 0.07 and 0.41 (4 shifts) or 0.02 and 0.21 (2 shifts) but need 17 to 19-bit words at −10 dBFS. Stylised and preliminary: energy constants are typed from memory of Horowitz (2014) at 45 nm and were not re-checked; the fidelity test uses synthetic voiced frames, not Speech Commands (unreachable), so no accuracy impact is measured.

A figure from its own output

Paper, code and results

• README and method
• Experiments and results
• Code
• Tests

More in Task-specific AI accelerators

• Does block floating point fix the 16-bit FFT scaling problem in an always-on MFCC front end?A shared data-chosen exponent for the 16-bit FFT in accel-frontend's MFCC path cuts cepstral error to 0.016 to 0.081 of the spread at -60 to -30 dBFS (one shift per stage: 0.35 to 0.72) for about +0.5% modelled energy, but no better than 18 to 20-bit fixed schedules on loud input (synthetic frames, no Speech Commands)
• Do the accel-* energy constants match Horowitz (2014), and does fixing the array-size term change accel-kws's conclusions?All 34 energy constants in the accel-* projects that have a counterpart in Horowitz's Fig. 1.1.9 are within 4% of it; re-costing accel-kws with the figure's array-size law (energy about the square root of size) moves energy to 0.23 to 0.81 of as built but keeps 17 of 18 front designs (Spearman 0.991)
• Does weight-sharing lookup-table inference beat an int8 MAC engine once small arrays cost less to read?Pricing every array by its own size flips accel-lut's negative result once read energy grows faster than size^0.26 (dense) or size^0.28 (pruned); Horowitz's slope is 0.5 above 8 KB but there is no point below it, so the verdict is conditional
• Is the OpenRAM access energy in SKY130 liberty files an energy at all, and does accel-pdkprice's weight-stationary verdict survive?OpenRAM's clk0 figure is an average power in mW over one cycle of an unstated period, not pJ: the unpacked verdict (register-file WS wins) holds for any period above 3 ns, while the packed verdict flips between 1 and 31 ns, so accel-pdkprice's packed conclusion is conditional
• Once the MFCC front end is counted, does the energy-optimal keyword model change?Joining accel-cascade's committed results with accel-frontend's 46.8 nJ per frame: cascades' classifier-only saving of up to 28% becomes at most 0.42% of whole-system energy, while the most accurate model (about 0.75 test accuracy) costs 23 to 26 times the cheapest classifier but only 5% more system energy (modelled, not measured)
• How big a keyword-spotting model fits a real Tiny Tapeout tile once the hidden layer goes past 64 units?Synthesising the streaming keyword-spotting engine at 96, 128, 192 and 256 hidden units (SKY130 HD, Yosys): area grows about 250 µm² per unit, so H = 128 (0.752 accuracy, 4-bit) needs a 2x2 tile (75,603 µm²) against a 1x2 (36,347 µm²) for H = 64 (0.737): +0.015 accuracy for twice the tile; H = 256 needs 4x2 at 50% utilisation, and its 8-bit layer-2 accumulator overflows 20 bits on a random model.

All 40 Task-specific AI accelerators projects →

Other research topics

Proper scoring & elicitation28Effort & peer prediction9Verification games & slashing19Detection & tolerances14Markets as routers13Prediction-market design9Paying contributors8Decentralised training systems23Digital twins & sim-to-real164LLM chip design loops40Robot manufacturing loops53Ambient AI assistant46Self-replicating manufacturing34Simulation-first design64Engineered clothing45Task-specific AI accelerators40Algorithmic chip-design loops42Open-source purpose-built robots44The software-engineering loop43Adversarial human imitation41Open-source residential construction41Autonomous multi-angle capture34Sensing, imaging and 3D31Energy supply and dense energy storage39Robot societies among people31Agentic co-design of low-cost robots27Laundry-hanging robots25Machine learning on qualitative data23Robots on the building site27Civilizations & group psychology1History & human sciences1

Produced with AI assistance (Claude). Results are from the code in this repository and are simulated or stylised unless the abstract says otherwise; nothing here implies affiliation with or endorsement by any lab, researcher or company cited. MIT licensed.