A United Airlines refund is not rent: red-teaming three tax calculators

24 faults seeded in three tools, 18 caught. A rent pattern with no word boundaries booked airline refunds and bank interest as rental income.

Today's red-team run seeded 24 faults into three files the repository's tests guard: the usage counter of Visit Keeper, the Rental Property Schedule E Categorizer and the Crypto Cost Basis Calculator. redteam.py campaign changes one operator or constant at a time, runs the file's tests and then restores the file.

FileSeededCaught beforeEquivalentCaught after the new tests
visit-keeper/usage.mjs8717 of 7
rental-property-schedule-e-categorizer/calc.mjs8508 of 8
crypto-cost-basis-calculator/calc.mjs8608 of 8

The real bug: rent matched inside other words

The Schedule E categorizer sorts every credit on a bank statement. A credit whose description matches its rent pattern goes on line 3, rents received. That pattern contained unit\s?\w+, lease and rental with no word boundaries, so:

CreditBooked asShould be
UNITED AIRLINES REFUNDrentnot categorised
COMMUNITY BANK INTERESTrentnot categorised
ESCROW SURPLUS RELEASErentnot categorised
RENTAL CAR REFUNDrentnot categorised

"United" and "community" each contain unit followed by a letter, and "release" contains lease. On a statement with a $300 airline refund and $12 of interest, the tool reported $312 more rental income than the landlord received. The tool's own rule is that a row with no clear category stays out of the totals until you say where it goes. These rows broke that rule without any warning.

The fix adds boundaries: \bunit\b, \blease and \brental(?!\s+car). "UNIT 4B RENT JAN", "LEASE PAYMENT 12 OAK" and "RENTAL INCOME OAK ST" still count as rent. The built-in sample statement categorises exactly as it did before. A new test holds four lookalike credits and four real rent credits.

What the tests missed

  • Rental income. No test had both rent and a late fee, so the tests still passed when rent + other_income became rent - other_income. A new test checks that $1,000 rent plus a $50 late fee comes to $1,050.
  • "Security" credits. A refund for a security camera could have been filed as a security deposit and nothing would have failed.
  • Cents. The dollar formatter could print three decimals unnoticed.
  • Crypto swaps priced per coin. A swap row with a price column and no total is valued at price times quantity. Dividing instead of multiplying passed every test.
  • Donating coins that lost value. If the coins are worth less than you paid, giving them away avoids no capital-gains tax. A mutant that reported $0.15 of "tax avoided" on a loss went through.

The one equivalent mutant is in Visit Keeper's storage guard. When the browser blocks site data, every later read and write is already wrapped in its own try, so the guard's return value changes nothing a visitor can see.

These tools give estimates from the numbers you enter. They are not tax advice.

All writing · More in Testing and red team · All tools and research