Data reachability: what this environment can actually fetch

Study · 2026-08-20 · portfolio/research/data-reachability/README.md

Measured 2026-08-20 by a scheduled run of the screening brief, which per CLAUDE.md's intake gate added no candidate. Reproduce with python3 probe_reachability.py; raw output in results-2026-08-20.json.

Why this was run

Four prior passes measured egress by probing a feed's canonical host — its API endpoint or its human-facing website — and marking the feed unreachable when that host was denied. #232 characterised the result as "a general non-registry-egress policy". #240 concluded that only mcp-supply-chain-watch could fetch its own runtime input data. Issue #139 §2d turned that into a strategy question for the account owner: general read-only outbound HTTP, or a portfolio that only builds registry-data businesses.

#250 then found a counter-example — npm advisory data is reachable as bulk and git distribution even though the OSV API is denied — and recorded the lesson as "prior passes tested API endpoints, not the git distribution of the same data." It applied that lesson to one candidate and stopped.

This pass tests it as a rule: for each feed, probe every distribution channel it publishes, not just the canonical one. A feed is reachable if any channel carrying the same data answers. That found two more reachable feeds and a materially different description of the policy.

The policy shape

The denial is not "non-registry". It is general-web hosts denied, code/package/object-storage infrastructure allowed:

Reachable (CONNECT succeeds)Denied at CONNECT
registry.npmjs.org, pypi.org, index.crates.io, proxy.golang.org (these bypass the proxy entirely — see noProxy in $HTTPS_PROXY/__agentproxy/status)example.com — the control
storage.googleapis.com, s3.amazonaws.comeur-lex.europa.eu, ec.europa.eu, publications.europa.eu, data.europa.eu, ted.europa.eu, euvd.enisa.europa.eu
raw.githubusercontent.com, api.github.com, github.com, codeload.github.comwww.w3.org, www.cisa.gov, www.federalregister.gov, services.nvd.nist.gov
gitlab.comapi.wordpress.org, epss.empiricalsecurity.com, huggingface.co

Two distinctions that earlier passes conflated, and that matter:

WebFetch is governed by the same policy

Untested until now, and worth closing because issue #139 §2c rests on it: the harness's own WebFetch tool is subject to the identical policy, not a separate route. WebFetch on eur-lex.europa.eu returns {"error_type":"EGRESS_BLOCKED"}; the same tool on raw.githubusercontent.com succeeds, so the failure is policy, not a broken tool. §2c's conclusion — that the pipeline's own graduation gate (verifying EU obligation dates against primary sources) is unsatisfiable under the current policy — holds through every route available to a session, not just curl.

Feeds that turned out to be reachable

OSV, at canonical scope — not just the npm slice

api.osv.dev is denied, but the canonical bulk distribution at osv-vulnerabilities.storage.googleapis.com answers, and it carries the whole database: 102 ecosystem prefixes, including all eight that matter to this portfolio.

Ecosystemall.zipLast modified (UTC)
npm219.97 MB2026-08-20 04:34
PyPI33.61 MB2026-08-20 04:19
Go11.24 MB2026-08-20 03:49
Packagist10.32 MB2026-08-19 20:08
Maven10.06 MB2026-08-19 21:48
RubyGems4.36 MB2026-08-18 20:34
crates.io3.38 MB2026-08-20 04:19
NuGet2.51 MB2026-08-18 18:19

Refreshed hours before this measurement, not a stale dump. Content checked rather than assumed — the npm archive holds 227,118 advisory records, and of a 400-record sample 399 carry a severity and 393 carry structured fixed-version ranges, which is the field SBOM matching actually needs. This is a materially richer feed than the registry.npmjs.org advisories-bulk endpoint #250 found, which is npm-only and returns a flat vulnerable_versions string with no cross-ecosystem coverage.

CISA KEV, via its git mirror

www.cisa.gov is denied; cisagov/kev-data on raw.githubusercontent.com answers with catalog version 2026.08.19, 1,671 entries, most recent addition dated the day before this run. That is the actively-exploited signal the CRA's Article 14 clock is keyed to, and nothing prior recorded it as available.

WCAG and axe-core rule definitions, via git

www.w3.org is denied; w3c/wcag and dequelabs/axe-core both serve over raw.githubusercontent.com.

Feeds still unreachable by any channel tested

EUVD/ENISA, EUR-Lex and every other EU institutional host, the US Federal Register, NVD, EPSS, TED, and api.wordpress.org. Each publishes only to its own website or API; none distributes to package, git, or object-storage infrastructure. No allowlist-free route exists for these, and searching for one is now a closed question rather than an open task.

Honest limits of this result