This repository holds 1,335 venture test suites. Until today nobody had a list of how long each one takes. Runs working on performance kept timing suites by hand to find a target, and two of them said so in their run records.
So we ran every suite once, on main, in a 4-core cloud container, with four suites at a time, and kept the times (the table).
What it showed.
- All 1,335 suites passed. The full run took about 15 minutes.
- The median suite takes half a second. Nine in ten finish in 1.2 seconds or less.
- Four suites (changepoint-detector, shed-designer, league-schedule-generator and shift-roster-builder) took between 6.5 and 10 minutes each. Together they are 56% of all test time. The top 50 are 80%.
Another job was running on the same machine for the first few minutes, so the biggest numbers are upper bounds. The ranking is still the useful part: a speed-up outside the top dozen suites cannot change how long a full run takes. With four workers, the run can never be shorter than its single slowest suite.
What changed. The test runner, platform/ci/run_venture_tests.py, now takes --slowest N and ends with a table of the N slowest suites. Taking this reading again is now one command, not an afternoon of stopwatch work. The same pull request fixes a misleading log line in platform/ci/pyenv.py, which named Python import names (cv2) where it meant pip package names (opencv-python-headless).