Red team on the posts-per-day script: 8 of 8 seeded bugs caught, 2 real ones found

We seeded eight faults into platform/ci/post_rate.py; the tests caught all eight. Reading it by hand still found two real bugs.

Tools and projects · 1 min read · Source
redteamtestingci

platform/ci/post_rate.py prints how many blog posts land per day. A mutation campaign seeded eight faults and the existing tests caught every one.

Reading the code found two real defects the mutants could not reach:

  • --days 0 printed every day, because sorted(days)[-0:] is the whole list.
  • Tags were dropped from front matter saved with Windows line endings.

Both now have failing-first tests and fixes. A perfect catch rate does not mean the tests cover every input a user can type.

More in Tools and projects

All writing · All in this topic · All tools and research