COVERAGE DROP CHECK · RUNS IN YOUR BROWSER

Did this PR drop your coverage?

Paste the Cobertura or JaCoCo XML from main (or the last release) and from this pull request. This page subtracts the two line-coverage rates and lists every file that got worse, better, new, or removed.

A floor such as --cov-fail-under=80 will let a PR drop from 92% to 81% and still pass. This check answers the delta, not the floor. Nothing is uploaded.

1. Get two coverage XML reports

Anything that writes Cobertura or JaCoCo XML works:

pytest --cov --cov-report=xml:baseline.xml     # then again on the PR for current.xml
coverage xml                                    # coverage.py, writes coverage.xml
nyc report --reporter=cobertura --report-dir .  # Istanbul / nyc
mvn test                                        # JaCoCo writes target/site/jacoco/jacoco.xml

Keep the baseline from the same suite and the same tool. Mixing a Cobertura report with a JaCoCo report is fine — both are reduced to line covered / line valid — but mixing different test sets is not.

2. Paste baseline and current

Paste both reports and click "Compare coverage". Default allowance is 0 — any drop fails.

Prefer a CLI, or want this as a CI gate?

Same logic, zero dependencies, one file:

python3 check.py baseline.xml current.xml            # exits 1 if coverage dropped
python3 check.py baseline.xml current.xml --max-drop 0.5
python3 check.py baseline.xml current.xml --json

Get check.py from the repo — stdlib only, nothing to install. It reads local files and makes no network call.

Common questions

How is a drop decided?

The overall line-coverage rate on the baseline report minus the same rate on the current report, in percentage points. If that number is greater than the allowance (default: any drop), the check fails. That is arithmetic on the two reports, not a guess and not a comparison against a fixed floor such as 80%.

Why not just use --cov-fail-under or a Codecov flag?

A floor such as pytest-cov's --cov-fail-under=80 lets a pull request drop from 92% to 81% and still pass. Codecov and Coveralls will catch a delta, but they need an account, a token, and an upload of the report. This tool answers the delta question from two files you already have, locally.

Which coverage formats does this work with?

Cobertura XML (coverage.py, pytest-cov --cov-report=xml, nyc/Istanbul --reporter=cobertura) and JaCoCo XML. Those are the reports most CI jobs already write. LCOV is not read yet.

Is my coverage report uploaded anywhere?

No. The browser page parses the XML you paste entirely in your own browser and never sends it anywhere. The CLI reads local files and makes no network call at all.

More: why coverage drops on a PR that "didn't touch tests", how this compares to a flat coverage floor like --cov-fail-under, or to Codecov, a hosted, connected alternative with a historical dashboard.

Related: if coverage looks fine but the same tests flip between CI runs, the flaky test finder classifies which tests are flaky versus broken from JUnit XML you paste — nothing uploaded.

Related: if every check fails in a few seconds with an empty body before you look at coverage, first check whether GitHub assigned a runner with the actions runner starvation check — paste the run’s job JSON and it reads runner_id.