Can an always-on assistant learn when to speak from accept/dismiss feedback alone, and what does it cost in dismissals?

What it found

A simulated assistant learns an interruption policy online (greedy, ε-greedy or Thompson-sampled logistic model) from the user’s reaction only when it speaks. With an uninformed prior none of the three learns a useful policy in 100 h; with a prior that the intent score is informative, greedy reaches 90% of an offline oracle’s rate after a median 22.5 h and about 185 dismissals, and Thompson sampling takes 32.5 h. It still earns 10.4 against the oracle’s 12.7 per hour, and after a drop in classifier quality it does far worse than frozen offline weights (0.3 vs 4.4 per hour). Stylised: independent windows, instant noise-free feedback, picked costs.

A figure from its own output

Paper, code and results

• README and method
• Experiments and results
• Code
• Tests

More in Ambient AI assistant

• Does inverse-propensity weighting fix an always-on assistant learning from its own accept/dismiss feedback?Does inverse-propensity weighting fix an assistant learning from its own accept/dismiss feedback: mostly negative, exploration cost dominates (stylised)
• Does selection bias actually hurt an always-on assistant that learns only from windows where it spoke?Does selection bias hurt an assistant learning only from windows where it spoke: yes, a lot in this world; a full-information fit reaches the oracle, weighting recovers part (stylised)
• Is the gap in an assistant that learns only from windows where it spoke caused by selection bias or by too few labels?Is the self-selected-feedback gap bias or label volume: mostly bias (a random sample of equal size nearly reaches the oracle); clipped weights help only with an informed prior (stylised)
• Does a doubly robust estimator fix learning from an assistant’s own self-selected feedback?Does a doubly robust estimator beat IPS on an assistant's self-selected feedback: no, a stylised negative result; its pseudo-labels carry 1/ε residuals and are worse than the biased fit except with an informed prior.
• Does capping the inverse-propensity correction rescue doubly robust learning from an assistant’s own feedback?Does capping the inverse-propensity correction rescue doubly robust learning from an assistant's own feedback: with an informed prior yes (the collapse at ε = 0.002 goes from 0.26 to about 12 utility/h), with an uninformed prior no; stylised, simulated only.
• Does cross-fitting the outcome model help capped doubly robust learning from an assistant’s own feedback?Does fitting the doubly robust outcome model out-of-sample (2-fold cross-fitting) help the capped correction: no, a negative result; differences are within seed noise or worse, and the unclipped collapse at ε = 0.002 persists (1.53 vs 0.26 utility/h); stylised, simulated only.

All 46 Ambient AI assistant projects →

Other research topics

Proper scoring & elicitation28Effort & peer prediction9Verification games & slashing19Detection & tolerances14Markets as routers13Prediction-market design9Paying contributors8Decentralised training systems23Digital twins & sim-to-real164LLM chip design loops40Robot manufacturing loops53Ambient AI assistant46Self-replicating manufacturing34Simulation-first design64Engineered clothing45Task-specific AI accelerators40Algorithmic chip-design loops42Open-source purpose-built robots44The software-engineering loop43Adversarial human imitation41Open-source residential construction41Autonomous multi-angle capture34Sensing, imaging and 3D31Energy supply and dense energy storage39Robot societies among people31Agentic co-design of low-cost robots27Laundry-hanging robots25Machine learning on qualitative data23Robots on the building site27Civilizations & group psychology1History & human sciences1

Produced with AI assistance (Claude). Results are from the code in this repository and are simulated or stylised unless the abstract says otherwise; nothing here implies affiliation with or endorsement by any lab, researcher or company cited. MIT licensed.