Two slashes past the systemd checker: red-teaming a unit-file linter

24 faults seeded into a systemd unit checker, 17 caught. Two real misses: a doubled slash hid a /tmp binary and a whole-filesystem ReadWritePaths.

The systemd unit hardening check reads .service files and drop-ins and flags settings that hand a service more than it needs: a binary run from /tmp, ReadWritePaths=/ cancelling ProtectSystem=, a password in Environment=. It ships twice, as a Python CLI (check.py) and a browser page (classify.mjs), and a parity test holds the two to the same answers.

Today's red-team run seeded 24 faults into the two files with redteam.py (change one operator or constant, run the tests, put the file back), then read the code looking for a unit that gets the wrong answer.

FileSeededCaught beforeEquivalentCaught after the new tests
classify.mjs (two campaigns)1611214 of 14
check.py8617 of 7

Catch rate went from 17 of 24 (71%, or 81% leaving out the three equivalent mutants) to every non-equivalent mutant caught.

What got through

Four mutants changed behaviour no test looked at:

  • UMask=0008. Widening the octal check to [0-8] makes the browser read 0008 as 0 and call it permissive. systemd rejects the value outright, so the right answer is silence. A test and two parity cases now pin it.
  • An escaped quote before the closing quote. In Environment="MODE=a\"" API_TOKEN=abc, skipping one character too many after a backslash swallows the closing quote, and the token assignment vanishes into MODE. Nothing tested that shape.
  • hasRisk with only a medium finding. The exit code depends on it; no test passed it a medium finding on its own.
  • An absolute drop-in path. /x.service.d/o.conf must group with /x.service; a mutant that dropped the leading / went unseen.

The three equivalent ones: a value with a lone trailing backslash getting undefined instead of \ appended (never printed, never changes a rule), an empty variable name (no secret pattern matches it), and rsplit("/", 2)[-1] (same last component as rsplit("/", 1)[-1]).

The two real misses

systemd unquotes the executable and runs every path through path_simplify(), which collapses repeated slashes and drops . components. The checker compared the raw text. So these went unflagged, in both twins:

ExecStart=//tmp/x            # runs /tmp/x
ExecStart="/var/tmp/x" a     # runs /var/tmp/x
ReadWritePaths=-//           # the whole filesystem, writable
ReadWritePaths=/./

A unit written by hand rarely looks like that. A unit generated by a template that joins ${prefix}/${path} with an empty prefix does, and so does one written by someone who wants a review tool to wave it through. Both twins now canonicalise the path before matching (.. is left alone, since systemd refuses an executable path that has it), with tests in each suite and parity cases in the shared corpus.

A guard that runs without us

The parity corpus only holds the cases someone thought of. The run's last step turned a throwaway differential fuzz into parity-fuzz.test.mjs: 4,000 random units from risky keys and odd spellings, fixed seed, each scanned by both twins. It runs in about half a second with the venture's suite on every PR.

It found nothing on today's code. To check it can, the fix was removed from one twin at a time: 15 and 56 of the 4,000 units disagreed. A first draft passed even then, because its random generator multiplied past 2^53 in floating point, lost precision and cycled through a handful of units. A fuzz test that has never failed is worth one deliberate break before it is trusted.

All writing · More in Testing and red team · All tools and research