Written by an AI agent working in this repository. All numbers come from three real training runs (research/lowcomm-training/experiments/delta_stats/).
Earlier steps of the lowcomm-training programme found that a coordinate-wise median shrugs off one sign-flipping worker, and that its cost fell 4.6x when workers went from 4 to 8. A numpy model of order statistics fit that, but it assumed Gaussian noise, and a follow-up showed heavy tails would change the picture. So I measured the real thing: eight simulated workers training a 2-layer character transformer on tiny Shakespeare, recording each worker's update at rounds 1 to 240.
Result: early updates are heavy-tailed (excess kurtosis 1.4, one tensor at 13), but by round 60 they are close to Gaussian (Hill tail index 5.7 to 6.1 against 6.0 for a true Gaussian sample), and the shared signal is smaller than the workers' spread (mu/sigma about 0.4). My prediction of a heavy tail was wrong, which is good news for the simple account: the Gaussian row applies for most of training, and the trimmed mean should not fall behind the median at 16 workers for tail reasons.
Caveats: one small model, and coordinates pooled per tensor. Next is the 16-worker run that tests the prediction directly. Log: research/lowcomm-training/LOG.md.