Does capping the inverse-propensity correction rescue doubly robust learning from an assistant’s own feedback?

What it found

A stylised follow-up to ambient-dr, which left clipping untested. Capping the correction weight at 1, 5, 20 or 100 (instead of 1/ε up to 500) removes the collapse with an informed prior: at ε = 0.002 the unclipped fit scores 0.26 utility per hour and the capped ones 11.7–12.2, against 10.6 for the biased fit and 12.71 for the oracle. With an uninformed prior it does not help: capped fits that use the clipped-IPS outcome model stay between −17 and −50, and those that use the prior as outcome model score 4–8 at best, still below the random-subset control (about 12). Simulated windows chosen by me; one cap grid, 20 seeds, nothing claimed about real users.

A figure from its own output

Paper, code and results

• README and method
• Experiments and results
• Code
• Tests

More in Ambient AI assistant

• Does cross-fitting the outcome model help capped doubly robust learning from an assistant’s own feedback?Does fitting the doubly robust outcome model out-of-sample (2-fold cross-fitting) help the capped correction: no, a negative result; differences are within seed noise or worse, and the unclipped collapse at ε = 0.002 persists (1.53 vs 0.26 utility/h); stylised, simulated only.
• Can the doubly robust cap be chosen by validation on an assistant’s own logs?Can the capped doubly robust cap be chosen by 4-fold IPS validation on the log: with an informed prior it matches the best fixed cap within seed noise, with an uninformed prior it does not rescue learning; stylised, simulated only.
• Does a doubly robust validation score choose the doubly robust cap better than an IPS score?A doubly robust validation score does not pick the DR cap better than IPS with the same 4 folds (equal with an informed prior, mixed with the uninformed one); 8 random folds with the DR score is best in all three uninformed cells but still far below the label-matched control (stylised).
• Which of score, fold count and fold type made cap selection better in ambient-drvalid?Ablating the three changes in ambient-drvalid (score, fold count, fold type): in the uninformed cells random folds help more than the DR score (IPS on 8 random folds is as good as DR on 8), no factor wins everywhere, and every selector stays far below the label-matched control (stylised, negative).
• Do repeated random fold assignments stabilise validation-based cap selection?Repeated random fold assignments for cap selection in the ambient-assistant world (follow-up to ambient-drablate): averaging 5 assignments helps the IPS-score selector at ε = 0.02 (3.5 → 5.3 utility/h) and nothing else beyond noise; stylised, simulated only.
• Does fitting only on the windows an assistant explored at random fix learning from its own feedback?Fitting only on the windows an assistant explored at random fixes the uninformed-prior failure of learning from its own feedback (11.99 vs −13.08 utility/h at ε = 0.02), at an exploration cost (stylised)

All 46 Ambient AI assistant projects →

Other research topics

Proper scoring & elicitation28Effort & peer prediction9Verification games & slashing19Detection & tolerances14Markets as routers13Prediction-market design9Paying contributors8Decentralised training systems23Digital twins & sim-to-real164LLM chip design loops40Robot manufacturing loops53Ambient AI assistant46Self-replicating manufacturing34Simulation-first design64Engineered clothing45Task-specific AI accelerators40Algorithmic chip-design loops42Open-source purpose-built robots44The software-engineering loop43Adversarial human imitation41Open-source residential construction41Autonomous multi-angle capture34Sensing, imaging and 3D31Energy supply and dense energy storage39Robot societies among people31Agentic co-design of low-cost robots27Laundry-hanging robots25Machine learning on qualitative data23Robots on the building site27Civilizations & group psychology1History & human sciences1

Produced with AI assistance (Claude). Results are from the code in this repository and are simulated or stylised unless the abstract says otherwise; nothing here implies affiliation with or endorsement by any lab, researcher or company cited. MIT licensed.