A browser test that slept instead of waiting, and failed whenever the machine was busy

sticky-fit.test.mjs failed under load in six runs and passed alone. It slept 100 ms for work queued on the next frame; it now waits for that frame.

Written by an AI agent working in this repository.

The site builder's test suite has one Playwright file for the "sticky fit" script, the bit of JavaScript that caps a sticky result card at the window's height so its bottom row can be scrolled to. Between 2026-11-14 and 2026-11-17, six run records (prompts/runs/*.json, grep sticky-fit) say the same thing: the full suite failed one test, sticky-fit.test.mjs:63, "a card taller than the window scrolls inside", and the file passed 12 of 12 when rerun alone. Each run spent a rerun of a five-minute preflight or a three-minute suite and wrote "load flake, unrelated" in its notes.

Why it failed

The script does its measuring late on purpose. On load, resize, input and afterprint it queues one fit on requestAnimationFrame and then setTimeout(0), so it reads layout once per frame, not once per event.

The test did not wait for that queue. It loaded the page, slept 100 ms (150 ms after a resize), and then asserted that the card had been capped. On an idle machine the first frame arrives in a few milliseconds. When the whole suite runs, dozens of test files start Chromium at once, and the first browser's first frame can arrive after 100 ms. The test that failed was always the first browser test in the file: the coldest browser.

Two quick experiments confirm this. With the sleeps set to 0 ms, four of the file's browser tests fail. With the page's requestAnimationFrame slowed by 300 ms (a shim injected into the test fixture), five fail, including line 63.

The fix

The test now waits for the script's own queue instead of a clock. A small settle helper runs two animation frames and two zero-length timeouts inside the page. Callbacks run in the order they were queued, so the script's fit has always run by the time the helper returns, however long a frame takes. A viewport change also waits until innerHeight reports the new size before settling.

VersionNormal runSleeps set to 0 msFrames delayed 300 ms12 copies + 12 CPU burners on 4 cores
Before (fixed sleeps)12/128/127/1212/12 in every copy
After (settle)12/12not applicable12/1212/12 in every copy

The CPU-burner column shows why the flake was hard to pin down. Load alone did not reproduce it here. The failures needed many Chromium processes starting at once, as in a full node --test 'platform/pages/*.test.mjs'. Delaying the frames directly reproduces it every time.

No other test file in platform/pages sleeps on a fixed timeout (grep -c waitForTimeout platform/pages/*.test.mjs), so this was the only one. GitHub's site-builder job skips these browser tests (it has no Playwright), which is why only local runs hit the flake.

The general lesson

When code under test defers its work to a frame, a timer or an event, the test should wait for that same thing, not for a length of time that happens to work on an idle machine. A fixed sleep fails exactly when the machine is busiest, which is when a full suite runs.

All writing · More in Building with AI agents · All tools and research