In a reverberant room, does PHAT win after all, and can the whitening exponent be chosen per window?

What it found

Adds simulated reverberation (exponentially decaying Gaussian tails, RT60 0–1.0 s, direct-to-reverberant ratio +5 to −5 dB, Schroeder's statistical model) to capture-realaudio's 10 real clips and tests GCC delay estimators for camera audio sync. Across 20 cells (10 rooms, SNR −5 and +5 dB, 200 windows each), PHAT is the worst fixed weighting in 19 of 20 cells: at RT60 0.6 s, DRR 0 dB, SNR +5 dB it puts 0.26 of 2 s windows more than one 30 fps frame off, against 0.09 for partial whitening with γ = 0.5 and 0.20 for plain correlation. Choosing γ per window by peak-to-sidelobe ratio beats PHAT but not the best fixed γ; an oracle selector would roughly halve the failures again. Limits: statistical rooms rather than measured impulse responses, white noise, sound-effect clips, simulated delay.

A figure from its own output

Paper, code and results

• README and method
• Experiments and results
• Code
• Tests

More in Autonomous multi-angle capture

• Does a soft confidence weight beat a hard switch for repairing stereo-depth view synthesis, and does matching cost help?Does a soft confidence weight beat a hard switch for repairing stereo-depth view synthesis, and does the matcher's cost help? A ramped weight on warp disagreement beats the hard gate by 0.1 to 0.5 dB in every condition (29.5 vs 29.3 dB held out at 15°) and leaves a quarter fewer seams, closing only a tenth of the oracle gap; matching cost is a worse signal than disagreement (level with hole filling, worse with a bystander). Toy scene.
• How much footage does a multi-camera travel kit produce, and what limits carrying and backing it up?How much footage does a multi-camera travel kit produce, and what limits carrying and backing it up? Four cameras make ~860 GB a day; cards weigh ~8 g but the evening copy takes 2.7 h on one reader, so a safe plan needs 6 readers and ~450 g (assumed device figures).
• How often must free-running cameras be re-synced by audio, and does rejecting wild estimates help?How often must free-running cameras be re-synced by audio? At 40 ppm a sync lasts 1000 s for a 40 ms tolerance; a drift gate on fresh estimates helps only if you re-sync every 300 s. Synthetic audio.
• Does a residual audio-sync error show at the cuts of an automatic multi-camera edit?Does a residual audio-sync error show at the cuts of an automatic edit? A sync made once leaves 50% of cuts jumping over a frame; a robust fit leaves none down to -10 dB SNR (2 s windows) and then fails outright, with 11 to 24% of cuts repeating or dropping a loud event.
• Does a robust fit or a Kalman filter hold a camera's clock better between audio re-syncs than a least-squares line, once the clock drift comes from a real crystal curve?Does a robust fit or a Kalman filter hold a camera clock better between audio re-syncs? On a clock rate computed from a sourced AT-cut crystal curve the plain offset hold is already fine up to 1800 s and every fit ties; only a Kalman filter tuned to an assumed 30 ppm wobble wins. Synthetic audio.
• Does finding the start offset first let short audio windows sync two drifting cameras at lower SNR?Does a coarse offset first let short windows sync drifting cameras? A 10 s window finds the offset, then 0.5 s windows hold within a frame from -12 dB SNR; plain 1 s windows fail to -5 dB. Synthetic audio.

All 34 Autonomous multi-angle capture projects →

Other research topics

Proper scoring & elicitation28Effort & peer prediction9Verification games & slashing19Detection & tolerances14Markets as routers13Prediction-market design9Paying contributors8Decentralised training systems23Digital twins & sim-to-real164LLM chip design loops40Robot manufacturing loops53Ambient AI assistant46Self-replicating manufacturing34Simulation-first design64Engineered clothing45Task-specific AI accelerators40Algorithmic chip-design loops42Open-source purpose-built robots44The software-engineering loop43Adversarial human imitation41Open-source residential construction41Autonomous multi-angle capture34Sensing, imaging and 3D31Energy supply and dense energy storage39Robot societies among people31Agentic co-design of low-cost robots27Laundry-hanging robots25Machine learning on qualitative data23Robots on the building site27Civilizations & group psychology1History & human sciences1

Produced with AI assistance (Claude). Results are from the code in this repository and are simulated or stylised unless the abstract says otherwise; nothing here implies affiliation with or endorsement by any lab, researcher or company cited. MIT licensed.