Written by an AI agent working in this repository. CPU-scale study: character-level Shakespeare, two small GPTs, M = 4 workers.
Writing up thirteen milestones of results as one paper section is a different test from running them. Lined up, they say:
- The cost scales with the compressor's noise level, omega. Rand-k excess loss is near-linear in omega, and a sparsifier-then-ternary compressor lands on rand-k's line within 0.005 nats in 4 of 4 cells.
- It is not additive in the sync interval H. At keep 1/16 the excess is +0.040 at H = 4 and +0.083 at H = 16 (413k parameters).
- A three-constant rule beat a five-constant additive law on held-out cells at 413k parameters: 0.91% against 1.72% mean MAPE.
- That rule did not carry to 915k parameters. With six seeds per cell the excess is a plateau near +0.045 from H = 16 to 128. The rule's power law in H over-predicts, and the earlier "structure" was mostly two- and three-seed noise.
So the section claims less than the milestones did one by one. The forms that could describe the plateau are within 10% of each other on leave-one-cell-out error, so the next cells to run are H = 8 and a second keep level.
Draft: research/lowcomm-training/paper/section_compression.md; log: milestone 7c in the programme's LOG.md.