What does the adaptive voice gate cost on a Cortex-M4, in flash and instructions per frame, and is the block-minimum floor worth it?
research/ambient-gatem4
What does the adaptive voice gate cost on a Cortex-M4? ambient-gatefw's C gate built with clang for Thumb-2 and run in an emulator: bit-exact, about 250 bytes of code, 3237 instructions per frame exact and 2096 with the block-10 floor, because the energy sum dominates. Instructions, not cycles; stylised.
What it found
A stylised follow-up to ambient-gatefw. Its C gate is built with clang for Cortex-M4 (Thumb-2, soft float) and run frame by frame in an emulator on ambient-gateroc’s synthetic audio. It is bit-exact with the Python reference, takes 236–258 bytes of code at -Os, and its only library call is the 64-bit divide, once per frame. The exact gate executes 3237 instructions per 20 ms frame at -Os; the block-10 floor executes 2096, a 35% cut, because the 320-sample energy sum (about 1980 instructions) is the floor under the saving. Instructions are not cycles, the divide is not counted, and nothing ran on hardware. Stylised.
A figure from its own output
Paper, code and results
More in Ambient AI assistant
- Can a noise-adaptive voice gate fit in about 120 bytes of RAM on a pendant microcontroller, and does a cheaper noise floor cost battery life?Can a noise-adaptive voice gate fit in about 120 bytes of RAM on a pendant microcontroller? C port of
ambient-gateroc's adaptive energy gate, bit-exact with the Python reference; a block-minimum noise floor cuts state from 1020 to 120 bytes and comparisons per frame from 249 to 24, moving modelled battery life by at most 0.07 h. Synthetic audio, host build only. - How much pre-roll buffer does an energy-gated audio pendant need to stop clipping the first word, and what does it cost in RAM and battery?How much pre-roll buffer does an energy-gated audio pendant need to stop clipping the first word? Synthetic audio: no gain in quiet rooms, in the loudest room onset recall 0.866 → 0.980 → 0.999 at 0, 5 and 25 buffered frames (0, 3.1, 15.6 KB) for 90 → 81 → 70 modelled hours; a too-deaf gate is not rescued (stylised)
- Does a fixed energy-gate threshold survive different rooms on an always-on audio pendant, and what does an adaptive one buy in battery life?Does a fixed energy-gate threshold survive different rooms on an always-on audio pendant? Synthetic audio, 20 seeds x 4 background levels: any fixed threshold that keeps speech recall >= 0.9 leaves the gate open all the time in the loudest room (57.6 h, no gating gain), while one adaptive threshold (4x a 5 s running-minimum floor) gives 82-90 h at recall 0.89-1.00 in all four; stylised, nothing measured on hardware
- What limits the battery life of a pendant-style always-on audio wearable?What limits battery life in a pendant-style always-on audio wearable: stylised nRF52840-class power model (mic floor ~170 h; speech-gated ~105 h vs 59 h always streaming) plus a 74-byte VAD gate that compiles for Cortex-M4; no hardware measured
- How much latency and memory does an open-model voice pipeline cost on a CPU, and what reply time does it leave?CPU voice pipeline (Silero VAD, Whisper tiny.en, Qwen2.5-0.5B, Piper) on synthetic speech: per-stage latency/RTF/memory and a reply-latency budget of 0.9–1.5 s at 200–800 ms endpoint hangover, ~0.7 s of it compute. Server CPU, not a phone.
- When should an always-on assistant speak, and does a policy tuned in a simulator keep its annoyance budget in a real room?When an always-on assistant should speak: wake, keyword, threshold and learned policies at equal annoyance, and what breaks when the simulator's room differs from the real one
All 46 Ambient AI assistant projects →
Other research topics
Proper scoring & elicitation28Effort & peer prediction9Verification games & slashing19Detection & tolerances14Markets as routers13Prediction-market design9Paying contributors8Decentralised training systems23Digital twins & sim-to-real164LLM chip design loops40Robot manufacturing loops53Ambient AI assistant46Self-replicating manufacturing34Simulation-first design64Engineered clothing45Task-specific AI accelerators40Algorithmic chip-design loops42Open-source purpose-built robots44The software-engineering loop43Adversarial human imitation41Open-source residential construction41Autonomous multi-angle capture34Sensing, imaging and 3D31Energy supply and dense energy storage39Robot societies among people31Agentic co-design of low-cost robots27Laundry-hanging robots25Machine learning on qualitative data23Robots on the building site27Civilizations & group psychology1History & human sciences1
Produced with AI assistance (Claude). Results are from the code in this repository and are simulated or stylised unless the abstract says otherwise; nothing here implies affiliation with or endorsement by any lab, researcher or company cited. MIT licensed.