When should an always-on assistant speak, and does a policy tuned in a simulator keep its annoyance budget in a real room?
research/ambient-interject
When an always-on assistant should speak: wake, keyword, threshold and learned policies at equal annoyance, and what breaks when the simulator's room differs from the real one
What it found
Simulated 10 s listening windows (needs at 3%, an intent score with d′ = 2, wake-phrase and keyword signals) compare interruption policies at equal annoyance. A wake word is precise (0.69) but catches 15% of needs; adding a keyword reaches 53% recall at 28.7 false interruptions per hour (0.72 for the wake word). At a budget of 1 false per hour a score threshold reaches recall 0.26 and a logistic policy that also uses the keyword and wake flag 0.33; recall at that budget climbs from 0.17 to 0.82 as d′ goes from 1 to 4. Tuned to 1 false per hour, policies keep the budget when only the need side shifts (weaker detector, rarer needs) but run at 3.6–3.9 per hour when the no-need background shifts (noisier score, more wake false alarms); 16 labelled hours restore it for the noisier score, and no threshold can for a wake detector that alone causes 3.5 false per hour. Stylised: independent windows, Gaussian scores, chosen costs, no real conversations.
A figure from its own output
Paper, code and results
More in Ambient AI assistant
- How much does speech-recognition error hurt an always-on assistant's trigger policies at equal annoyance?How much speech-recognition error hurts an always-on assistant's trigger policies at equal annoyance: from word error rate 0 to 0.5 a two-word wake phrase loses 74% of its recall and a transcript-reading threshold 76%, an acoustic-only score about 54%, and the transcript policy falls behind the acoustic one at WER 0.3 to 0.4 if its separation shrinks as (1-WER)^2 (stylised).
- When moments of need cluster in time, does a cooldown, a boost or a back-off help an always-on assistant at equal annoyance?When moments of need cluster in time, does a cooldown, boost or back-off help an always-on assistant at equal annoyance: an episode-aware boost gains up to 0.05 recall under strong clustering, a cooldown always loses recall, back-off is a wash; stylised simulation.
- Under a storage limit, which transcripts should an on-device assistant keep, and what survives?What an on-device assistant should keep from a week of simulated transcripts: BM25 retrieval, extractive summaries and byte-limited retention policies (stylised)
- Which parts of an always-on Android listening service can be tested without a phone?Testable Kotlin core of an always-on listening service (re-framing, energy VAD, utterance segmenter, headset-drop policy; 7 tests pass under a shim) plus an unbuilt Android foreground-service sketch; no SDK, no device numbers
- When is a glance better than speech for an always-on assistant?When a glanceable display beats speech for an always-on assistant: a stylised utility model over noise, company, free eyes and message length, with sensitivity and twin-gap checks
- Does urgency, and a chime-plus-glance mode, change how an always-on assistant should deliver a message?Urgency classes and a chime-plus-glance mode added to the glance-vs-speech utility model for an always-on assistant: a stylised model with sensitivity and twin-gap checks
All 46 Ambient AI assistant projects →
Other research topics
Proper scoring & elicitation28Effort & peer prediction9Verification games & slashing19Detection & tolerances14Markets as routers13Prediction-market design9Paying contributors8Decentralised training systems23Digital twins & sim-to-real164LLM chip design loops40Robot manufacturing loops53Ambient AI assistant46Self-replicating manufacturing34Simulation-first design64Engineered clothing45Task-specific AI accelerators40Algorithmic chip-design loops42Open-source purpose-built robots44The software-engineering loop43Adversarial human imitation41Open-source residential construction41Autonomous multi-angle capture34Sensing, imaging and 3D31Energy supply and dense energy storage39Robot societies among people31Agentic co-design of low-cost robots27Laundry-hanging robots25Machine learning on qualitative data23Robots on the building site27Civilizations & group psychology1History & human sciences1
Produced with AI assistance (Claude). Results are from the code in this repository and are simulated or stylised unless the abstract says otherwise; nothing here implies affiliation with or endorsement by any lab, researcher or company cited. MIT licensed.