Which of score, fold count and fold type made cap selection better in ambient-drvalid?
research/ambient-drablate
Ablating the three changes in ambient-drvalid (score, fold count, fold type): in the uninformed cells random folds help more than the DR score (IPS on 8 random folds is as good as DR on 8), no factor wins everywhere, and every selector stays far below the label-matched control (stylised, negative).
What it found
A stylised, mostly negative ablation of ambient-drvalid: all 8 combinations of validation score (IPS or doubly robust), folds (4 or 8) and fold type (contiguous or random) choosing the doubly robust cap, 20 seeds. With an informed prior all eight are within 0.2 utility/h. With the uninformed prior, random folds are the most consistent help: at ε = 0.02 the 4-fold IPS selector gives −1.4 (std 24) against 5.6 (std 7.3) with random folds, and IPS on 8 random folds (6.3) matches DR on 8 random folds (5.9). At ε = 0.005 the DR score is worse than IPS on contiguous folds (−26.9 vs −17.7) and no selector is clearly best; seed standard deviations are 5–55. Every selector stays far below the label-matched control (about 12). Simulated, independent windows, no significance tests.
A figure from its own output
Paper, code and results
More in Ambient AI assistant
- Do repeated random fold assignments stabilise validation-based cap selection?Repeated random fold assignments for cap selection in the ambient-assistant world (follow-up to
ambient-drablate): averaging 5 assignments helps the IPS-score selector at ε = 0.02 (3.5 → 5.3 utility/h) and nothing else beyond noise; stylised, simulated only. - Does fitting only on the windows an assistant explored at random fix learning from its own feedback?Fitting only on the windows an assistant explored at random fixes the uninformed-prior failure of learning from its own feedback (11.99 vs −13.08 utility/h at ε = 0.02), at an exploration cost (stylised)
- How many false wakes a day does an always-on trigger get when its frame scores are correlated, and what does requiring m frames in a row buy?How many false wakes a day does an always-on trigger get when its frame scores are correlated? Exact grid-chain rates for "m frames in a row" on AR(1) scores, checked by Monte Carlo: a threshold set by the independent-frames rule gives 613 / 4,793 / 20,993 wakes a day (not 1) for m = 2 / 3 / 5 at phi = 0.9, while a single-frame threshold is nearly unaffected. Stylised.
- How bursty are an always-on trigger's false-wake counts, and does a lockout help?How bursty are an always-on trigger's false-wake counts? On AR(1) frame scores the Fano factor is near 1 (1.05–1.5), but a 5% window-to-window drift in noise scale lifts it to about 9 for m = 3 (38 at 10%); a 10 s lockout cuts it to 2.5 but also cuts the mean by a third. Stylised.
- How does the burstiness of an always-on trigger's false-wake counts grow with the counting window when the noise level drifts for hours?How does the Fano factor of false-wake counts grow with the counting window when the noise level drifts for hours? At 5% persistent drift (about 3 h correlation time) it grows from 1.9 at 100 s to about 150 at 28 h, against a flat 1.9 for independent drift. Stylised; the drift is assumed.
- Do a dual-MAC energy loop and a divide-free threshold make the Cortex-M4 voice gate cheaper?Do a dual-MAC energy loop and a divide-free threshold make the Cortex-M4 voice gate cheaper: 17-37% fewer instructions with the block-minimum floor, slower without it, RAM doubled; decisions unchanged (emulator counts, synthetic audio)
All 46 Ambient AI assistant projects →
Other research topics
Proper scoring & elicitation28Effort & peer prediction9Verification games & slashing19Detection & tolerances14Markets as routers13Prediction-market design9Paying contributors8Decentralised training systems23Digital twins & sim-to-real164LLM chip design loops40Robot manufacturing loops53Ambient AI assistant46Self-replicating manufacturing34Simulation-first design64Engineered clothing45Task-specific AI accelerators40Algorithmic chip-design loops42Open-source purpose-built robots44The software-engineering loop43Adversarial human imitation41Open-source residential construction41Autonomous multi-angle capture34Sensing, imaging and 3D31Energy supply and dense energy storage39Robot societies among people31Agentic co-design of low-cost robots27Laundry-hanging robots25Machine learning on qualitative data23Robots on the building site27Civilizations & group psychology1History & human sciences1
Produced with AI assistance (Claude). Results are from the code in this repository and are simulated or stylised unless the abstract says otherwise; nothing here implies affiliation with or endorsement by any lab, researcher or company cited. MIT licensed.