Fifty-nine federal agencies are validated against the schema that exempts them

What we measured, and from where

Four public surfaces, all keyless, all pinned to 21 August 2026. The data.gov harvester API at harvest.data.gov, which publishes 793 harvest sources, 120 organizations and 56,764 harvest jobs, and which describes itself in an OpenAPI document at /openapi.json. The publishers' own data.json catalogues, of which we fetched 136 and parsed 97. The DCAT-US 1.1 schemas and JSON-LD context served from resources.data.gov, together with the authoritative OMB bureau code register of 368 agency and bureau pairs. And the harvester source itself at GSA/datagov-harvester, where the schema selection logic sits atharvester/harvest.py lines 158 to 162.

The measurement vantage point was London. That detail is not decoration, and the section on what went wrong explains why it nearly produced a false finding.

Finding one: the binding, not the data, decides what is checked

When an agency registers a catalogue with data.gov, the registration carries a schema type. The harvester reads it and picks a validation schema: dcatus1.1: federal loadsfederal_dataset.json, anddcatus1.1: non-federal loadsnon-federal_dataset.json. That choice is made once, at registration, and it decides for good which rules the catalogue will ever be measured against.

We diffed the two schema files as published. They are the same document with two exceptions, both confined to the two federal fields. In the federal schema, bureauCode andprogramCode appear in the required array. In the non-federal schema they do not, and both additionally gain a branch permitting an explicit null. Nothing else differs.

Then we cross-tabulated the registrations against the organization types data.gov itself publishes. Every one of the 76 federal-typed sources is bound to the non-federal schema. The 59 organizations behind them include the Department of Defense, the Department of the Treasury, the Social Security Administration, NASA, the FDIC, the Securities and Exchange Commission, the Library of Congress, the General Services Administration, and the Office of Management and Budget. OMB issues the bureau code requirement. GSA operates data.gov. Both are registered under the schema that does not check it.

This is the shape of defect we have found in every register family we have audited, and it is why we call the practice register assurance rather than data quality. Assurance stops at the register boundary. Each catalogue is validated diligently against the rules it was pointed at, and nothing anywhere checks whether it was pointed at the right ones. The failure is invisible from inside either system, because from inside both, everything passes.

Finding two: the rule that would have caught it does not work either

The obvious remedy is to rebind those 76 sources to the federal schema. We checked what that would actually enforce, and the answer is close to nothing. The bureau code pattern is published as[0-9]{3}:[0-9]{2}, with no anchors. JSON Schema patterns are unanchored by default and match on substring, soxx024:01yy validates cleanly. So does422:00:00, and so does every malformed value we found in production.

Worse, the property contradicts itself. The description beside that pattern gives015:010 as the normative example, which is three digits, a colon and three digits. The pattern requires three digits, a colon and two digits. A publisher who reads the documentation and follows the example produces values the pattern is supposed to reject. That is exactly what the Department of Homeland Security does, on 1,168 datasets, with values such as024:010 where the OMB register lists bureau 10. On the evidence, DHS is not the party in error here.

In fairness, this is confined to one property. We checked programCode, whose example 015:001 is consistent with its own pattern. The defect is specific and therefore fixable in one line.

Finding three: what the unchecked field actually contains

Across 79,413 bureau code values on 79,404 datasets from 49 federal catalogues, 76,498 are well formed and present in the OMB register. The federal estate is not in ruins. But 2,246 fail an anchored read of the pattern, and 669 are well formed while naming nothing in the register that issues them.

The clearest single case is the Department of Energy, which publishes bureau code018:001 on 997 datasets. Agency 018 in the OMB register is the Department of Education. The Department of Energy is 019. Anyone rolling up federal open data holdings by publishing agency, which is the ordinary use of the field, attributes those 997 datasets to the wrong cabinet department.

The second pattern is the placeholder. 000:00 is not in the OMB register and never has been, yet the Department of Labor uses it on 401 of its 404 datasets, the General Services Administration on 103, the Department of Energy on 66, and the Federal Deposit Insurance Corporation on all 23 of its own. It matches the pattern perfectly. It identifies nothing. This is the same failure mode as the charity register placeholders we documented in the UK, arriving independently in a different country and a different sector, which is what makes it a defect class rather than an anecdote.

Finding four: the aggregator reports success while ingesting nothing

data.gov publishes its own harvest telemetry, and it is unusually candid. Taking each of the 793 sources and reading its most recent completed job, 179 of them, 22.6 per cent, errored every single record they saw. The status field on those runs reads complete. A source that ingests nothing and a source that ingests everything are reported the same way to anyone reading the status alone.

Behind that sits a corpus of 4,176,301 validation error rows, exposed without an API key, which we can find no evidence of anyone aggregating. The individual rows are humbling in an ordinary, human way. A licence field containing text-align:Left;, a fragment of CSS that escaped a content editor. A contact address readingmailto:jaeger,stephanie<sjaeger@sandiego.gov>, an Outlook display name pasted whole. Another readingmailto:patricia.chardon-at-caricoos.org, where an anti-spam habit has quietly destroyed the address. A landing page given aswww.woodsholegroup.com with no scheme, so it is not a URI at all. None of these people did anything unreasonable. The systems around them simply never told them.

The method, and what it costs to be sure

The artefact is an OWL 2 ontology with a SKOS scheme registry and three SHACL layers, one shape per defect class, so that a validation report is a findings table rather than a wall of constraint violations. The modelling commitment is that conformance is never a property of a dataset. It is an assertion by a named source carrying a date, which makes disagreement representable and makes silence visible: a validator that was never pointed at a rule cannot observe its breach, and that silence is itself a position.

Every headline is computed twice, set-based in Python and again in SPARQL over the 960,914-triple evidence graph, by a script that exits non-zero if the two ever disagree. All nine agree. SHACL is a third path and it corroborates the load-bearing number independently: layer 3 reports exactly 76 violations, the same 76 sources. The vocabulary and shapes are additionally validated and linted with our own open-source engine.

That machinery earns its keep by catching us. Three claims in this work were wrong before they were right. We first reported that the two schemas differ in exactly one respect; the offline test pinning the claim failed, and the true answer is two. We first reported that 17 harvest sources were broken because they returned HTTP 403; they were not, and the section below explains why. And we first excluded a date defect as untraceable because we were grepping the source JSON for a value that only exists after parsing. All three corrections are recorded in the build report rather than quietly applied, because a build report that only records successes is not a build report.

The finding that died, and why we are telling you

Seventeen registered harvest sources refused us, including every one of the Department of the Interior's twelve bureau catalogues, the Social Security Administration, the Department of Defense and the Department of Agriculture. All 17 returned HTTP 403 to our research client, and all 17 returned 403 again to a standard browser user agent, which ruled out the easy explanation. It looked like a substantial finding about the reachability of mandated public data.

It was not. Every one of those responses came from an Akamai or CloudFront edge node in London, which is where we were measuring from. The way to settle it was to stop measuring and go and read data.gov's own job history, which is public: the Social Security Administration source completes daily, ingests 2,363 records and reports zero errors. The 403 is a fact about our vantage point, not about the source. No defect is claimed, the hypothesis is recorded as dead, and separating edge denial from genuine unavailability would need a US vantage point we do not have.

We report this at length because it is the most useful thing in the study for anyone doing similar work. A confident, well-evidenced, entirely false finding was one unchecked assumption away from publication.

What follows from this

Three of these defects are one-line fixes in public repositories, and none of them requires an agency to change how it publishes anything. Anchor the bureau code pattern. Reconcile the property description with the pattern printed beside it. Rebind federal publishers to the federal schema, in that order, because rebinding first would enforce a rule that currently enforces nothing.

For a publishing agency the question is narrower and more immediate: how many of your own datasets carry a bureau code that names nothing, and would you know? For the 59 organizations named in this study the answer is currently no, because the system that would tell you is not looking.

Prior art and credit

The DCAT-US 3.0 tiger team inherited a website, a JSON Schema and a SHACL definition that were out of sync, with no source of truth and a three-month deadline, and said so publicly. Nothing here is a criticism of that work. The 1.1 profile measured in this study predates it, and the defects are inherited rather than introduced. data.gov also deserves credit for publishing the telemetry that made most of this possible: very few aggregators expose their own failure rates without an API key, and the study would not exist if it did not.

The artefact

The ontology, the SKOS registry, the three SHACL layers, the pipeline, the verified queries and the build report are public. Code is MIT, the ontology and documentation are CC BY 4.0. The build report states what was fetched, what was verified, what was sampled and with which seed, which hypotheses died, and what could not be obtained. A companion study profiles DKAN, one of the portal platforms in this estate.