MRR Pulse turns a Stripe account, or a Stripe subscriptions CSV export, into a monthly recurring revenue history: MRR, new and churned revenue, cohorts and a run-rate forecast. This red-team run attacked its metrics code, and the SQL migration lock checker alongside it.
The seeded faults
Ten small faults went into the MRR maths and eight into the lock checker's engine, one at a time, with each product's own tests run after every one. 13 of the 18 broke a test.
Of the five that got through, three could not change any output (an unreachable ternary branch, a length check a later check repeats, a flag that is always reset before it is read). Two were real holes:
- The customer-concentration figure sorts subscriptions largest first. Every test fed it values that were already in that order, so a broken sort comparator passed. A test with shuffled input now catches it.
- In the lock checker, a stray
ROLLBACKbefore anyBEGINrestores "the timeout state at BEGIN". Seeding that state as "on" silenced every later finding, and nothing noticed. A test now pins it.
After the two tests, all 18 seeded faults are caught.
The real bug
Stripe has two statuses for a subscription whose first payment failed: incomplete, and, after about 23 hours with no successful payment, incomplete_expired. Neither was ever revenue. The dashboard treated incomplete_expired as a subscription that had lived and ended:
- From the CSV export, the parser kept an end date only for
canceledrows. Anincomplete_expiredrow lost its end date and then counted as live MRR every month from its creation onward. In the reproduction, one $500 row that never paid sat in the history for seven months, so the backfilled MRR read $600 against a live figure of $100. - From the API, Stripe gives the subscription an
ended_ata day after creation. When those 23 hours crossed a month end, the month showed the amount as new MRR and the next as churn.
The fix treats both statuses as never paid in the history and in the signup cohorts, with a failing test for each path written first. The README's "Not counted" list now names them.
The general point for anyone computing MRR from Stripe: filter on whether money arrived, not on whether the subscription object has an end date. A status list that says what ended does not say what paid.