What 13 Million Triples Reveal About the Quality of US Federal Vocabularies

An ontology that does not parse, and the 7,782 concepts that depend on it

The Library of Congress MADS/RDF ontology, served at id.loc.gov/ontologies/madsrdf/v1.rdf, fails to parse at line 340. The construct is an rdf:Description node element carrying an rdf:resource attribute, appearing as a child of an owl:unionOf with rdf:parseType="Collection". It occurs 23 times.

The RDF 1.1 XML Syntax grammar is explicit here. The nodeElement production permits only rdf:ID, rdf:nodeID or rdf:about, while rdf:resource is valid on an empty property element. A collection parse type contains a node element list, so its children are node elements. The intended attribute is almost certainly rdf:about.

Two independent parsers agree that something is wrong and disagree instructively about what to do. rdflib refuses the file. OWLAPI, via ROBOT, does not fail: it emits a recognition warning and silently substitutes error entities named Error1 through Error8. A pipeline built on the second behaviour produces a graph that looks complete and quietly is not, which is how a defect of this kind survives for years without anyone filing anything.

The construct appears zero times in the BIBFRAME and PREMIS 3 files, so this is specific to MADS/RDF rather than a pattern across id.loc.gov. It matters beyond one file because the Thesaurus for Graphic Materials, a substantial Library of Congress vocabulary, types its 7,782 concepts with madsrdf:Topic. A live vocabulary is typed against an ontology that no conforming parser will read.

A vocabulary of language codes that carries no language tags

Every one of the 507 concepts in the Library of Congress ISO 639-2 vocabulary violates SKOS integrity condition S14, which states that a resource has no more than one value of skos:prefLabel per language tag.

The cause is not carelessness about labels. It is the opposite. Each concept is given three preferred labels, in English, French and German, which is more multilingual care than most vocabularies manage. None of the three carries a language tag. Three untagged labels are three labels sharing the same absent tag, so the condition is violated 507 times out of 507. The concept for the Banda languages carries "Banda languages", "banda, langues" and "Banda-Sprachen (Ubangi-Sprachen)", and a consumer has no machine-readable way to tell which is which.

This is the class of defect a checker finds in a second and a careful human reader never notices, because on the screen the labels look correct and the multilingual intent is obvious. It is also a one-line fix per label, in the vocabulary whose entire subject matter is which language something is in.

The best-governed asset in the sample is the one we can prove is two years old

The NAL Agricultural Thesaurus passes the governance checks that most of this sample fails. It declares its licence, Creative Commons Attribution 4.0, inside the payload. It names its publisher. It carries dated metadata. On the three fields where 21, 20 and 14 assets respectively fall short, NALT is clean.

That is precisely why the next paragraph is possible.

NALT declares dcterms:modified 2024-07-16. Every file in the download directory at lod.nal.usda.gov/downloads, covering all four subschemes across N-Triples, RDF/XML, Turtle and MARC, returns a Last-Modified of 24 July 2024. The Turtle payload inside the archive is named nalt-full_dwn_20240716.ttl. Re-fetched on 25 August 2026, it is byte-identical to the 16 August 2026 snapshot, SHA-256 beginning a4a72e8f03aa. The service still describes itself as NALT 2024.

There is public context for that. Federal procurement records show the Agricultural Research Service contracting for VocBench, the open source vocabulary editing platform managed by the EU Publications Office, specifically for the NAL Thesaurus, on an award running from September 2021 to 29 September 2026, alongside separate consulting awards under the same "NALT for the Machine Age" programme. A vocabulary being re-platformed is a vocabulary whose published output can sit still for a long time.

The point for a register is narrower than the reason. The question is answerable at all only because NALT declares a date, publishes a licence and serves stable bulk files. For the 14 assets in this sample that carry no version or dated metadata, the same question cannot be put. An asset that never claimed a date cannot be shown to have gone stale, and a consumer has no way to distinguish a vocabulary that is finished from one that is abandoned.

That is the argument for the conventional column in one example. None of licence, publisher and version breaks a rule. All three are what makes decay observable.

NALT carries the register's other S14 finding as well, and running it to ground turned a count into something actionable. Thirty-two of its 76,691 concepts carry two skos:prefLabel values in Spanish. Twenty-eight are the same scientific name twice with different capitalisation, Callithrix alongside callithrix. One is a misspelling rather than a duplicate, calimico for Callimico. Two are a singular beside a plural, rana and ranas. One is an acronym variant. The English side is clean throughout.

All thirty-two sit in the NALT AWIC subscheme, which holds 887 concepts in the file. None of the other 75,800 concepts is affected. That places the defect in one load rather than in editorial practice across the thesaurus, which is a materially different thing to tell a maintainer, and the per-concept list is in the repository.

The rest of the normative column, and what is absent from it

The remaining specification violations are few and concrete. Four NASA GCMD concept scheme endpoints return application/xml rather than application/rdf+xml, so a client negotiating content properly cannot tell it has been handed RDF, although the payload is well formed and parses. Three assets carry pairs of concepts asserted both associatively and hierarchically, which SKOS condition S27 makes disjoint: 4,700 of roughly 246,000 pairs in the USGS Common Geographic Areas file, which at that scale reads as a generation artefact rather than editorial slips, 6 of 32 in GCMD Platforms, and 1 of 1,106 in the USGS Thesaurus. Two assets breach S14 on a small proportion of concepts.

There were no failures on retrievability: every asset was obtainable at the address its publisher advertises. There were no failures on transport security: every plain HTTP request either redirected to HTTPS or, in the EPA's case, was refused outright, which is stronger than the policy requires. There were no inconsistent ontologies, no unsatisfiable classes, and no violations of SKOS conditions S9, S13, S37 or S46 anywhere in the sample. Federal publishers are not shipping broken logic. They are shipping unlabelled artefacts.

Governance is the finding, not the footnote

Twenty-one of 28 assets carry no licence statement anywhere in the payload. Twenty name no publisher or creator. Fourteen carry no version or dated metadata. Thirteen have terms without definitions, and twelve do not return RDF when RDF is requested at their namespace.

None of that breaks a rule. All of it decides whether a downstream consumer can act. An organisation building a retrieval pipeline over federal vocabularies has to answer three questions before it ships: may we redistribute derived data, which version did we ingest, and how do we find out when a term changes underneath us. For most of this sample the payload answers none of the three, and the answers live on a landing page that no machine will read. A licence on an HTML page is not a licence a pipeline can act on.

There is also a coverage story that belongs in the findings rather than in a caveat. Several assets sought for this register are not available as RDF at all. NIEM 6.0, the largest US federal data exchange standard, publishes its normative model as XML Schema, and the only RDF-adjacent artefact in its repository is a JSON-LD context of 5,418 bytes. The EPA Substance Registry Services returns HTML when RDF is requested. NIST OSCAL is XML and JSON Schema. Those are recorded as sought and not obtainable rather than scored against criteria they were never built to meet, because a register that lists only what worked is not a register of the estate.

Why these findings survive contact with a publisher

Tools that score ontologies have existed for over a decade and are not much used by the people who publish ontologies. The reason is not indifference to quality. It is that most reports lose the argument the first time a publisher pushes back, and they lose it in three predictable ways: convention presented as standard, so a publisher reads one item about missing textual definitions, correctly notes that no W3C Recommendation requires them, and stops reading, which costs the twenty findings that were real; inapplicable presented as failed, so a SKOS concept scheme is marked down for carrying no OWL axioms it was never meant to carry; and an unreproducible subject, so a report says the NASA keywords have some number of problems without saying which retrieval, on what date, from which endpoint.

This register is built to lose none of those three arguments. Every check declares normativity, which is a claim about authority. A check is normative only if failing it violates a published specification, and it must name the clause. Ten of the 26 qualify. Six are the SKOS integrity conditions S9, S13, S14, S27, S37 and S46, stated normatively in the SKOS Reference. The others are RDF syntax conformance, media type correctness and OWL 2 consistency. One cites government policy rather than a standards body: OMB Memorandum M-15-13 requires federal websites to serve only over HTTPS, which makes transport security a compliance question for these publishers specifically.

Every check separately declares severity, which is a claim about consequence, and the two axes are allowed to diverge in both directions. An undeclared licence is conventional, because no specification compels a publisher to state a licence inside the payload, and high severity, because a consumer who cannot determine the licence cannot lawfully redistribute derived data. A mislabelled media type is normative, because it violates the media type registration, and low severity, because almost every client sniffs the content anyway.

Findings are W3C EARL assertions rather than a bespoke result format, because EARL already carries the distinction this domain needs. It has five outcomes, not two, and this register uses them in earnest. A SKOS scheme carrying no OWL axioms is recorded earl:inapplicable for OWL profile conformance, not earl:failed. A reasoner that exhausts its budget yields earl:cantTell, never a silent pass. Of the 728 results in this run, 155 are inapplicable and 50 could not be determined, and publishing those counts is what lets a reader recompute the aggregate under a different weighting.

Assessments attach to a snapshot, never to an abstract asset. Each snapshot records the URL requested, the URL finally resolved to, the HTTP status, the media type actually returned, the byte count and a SHA-256 of the bytes. A publisher can re-fetch and compare hashes. If the hash changed, the finding may be stale and this register is wrong to keep asserting it. That property is what turns a report into something contestable, and it is also what made the NALT staleness measurable a week later without re-running anything.

Every headline figure on this page was computed twice, once set-based in Python over the results and once by SPARQL over the EARL graph, by a script that shares no code between the two paths and exits non-zero if they disagree. They agree on every figure.

The catalogue in full

Every check ran against all 28 assets, which is where the 728 results come from. The table below is the whole instrument, with the authority each check derives from, its severity, and the number of assets that failed it in this run. The two blocks are the distinction the register is built around, and the totals are 10 against 120.

Reading down the conventional block is the fastest way to see the shape of the estate. The five largest counts are licence, publisher, version, definitions and content negotiation, in that order, and none of them is a logic problem.

Where this sits against prior work

Reproducible ontology quality tooling is not new. Mader, Haslhofer and Isaac defined fifteen computable quality functions, implemented them as qSKOS, and ran them over fifteen vocabularies including LCSH, MeSH and NAICS, published at TPDL 2012; they found issues in all fifteen, including 342,848 undocumented concepts in LCSH, and the SKOS checks here are a narrower and more conservative descendant of theirs. OOPS! has catalogued 41 ontology pitfalls from an empirical analysis of more than 693 ontologies since 2014, FOOPS! has scored vocabularies against the FAIR principles since 2021, and the OBO Foundry dashboard runs a standing monthly report across roughly 190 ontologies.

The gap those leave is a national one. The OBO dashboard reaches exactly two US federal vocabularies, and only because biomedicine adopted OBO. The last systematic look at federal SKOS covered three vocabularies and was published fourteen years ago. What is new here is a standing, contestable register pointed at one government's semantic estate, which separates specification violations from community conventions, publishes the snapshot hash behind every claim, and gives the assessed publishers a documented right of reply.

What happens next, and what the publishers say

Notices are going out to the publishers named on this page, each carrying the retrieval, the clause and the per-concept list where one exists. Where a publisher is content for its response to be recorded, it is logged in the repository under a publisher-response label, whether it confirms a finding, corrects it or rejects it. Correspondence is not published without the publisher's agreement.

Three responses are equally useful. If a finding is wrong, showing the retrieval or the check we got wrong gets the register fixed and re-run, with the correction recorded publicly rather than quietly edited. If a finding is right and already fixed, the next run picks it up, and because snapshots are hashed the improvement is visible as a changed hash and a changed outcome rather than as a claim. If a check simply does not apply to an asset, that is a defect in the catalogue rather than in the asset, and the fix is a narrower applicability rule or an outright removal.

The register is built to be re-run on a schedule for that reason. A first pass says what the estate looks like. A second pass, against the same hashed baseline, says which publishers moved.

Scope, and two findings that were thrown away

Coverage is a seeded sample rather than a census, biased toward agencies that publish RDF at all, which by construction underrepresents the agencies whose semantic assets are least accessible. The Library of Congress accounts for 15 of the 28 assets, so proportions are proportions of this sample. Retrievability was observed once, from one network location, on one day, which is the main reason the register is built to be re-run rather than published once. Reasoning is bounded by a time budget, and assets that exhaust it report that they could not be determined. Assets typed with MADS/RDF rather than SKOS, such as the Thesaurus for Graphic Materials, fall outside the SKOS battery and are assessed only on retrieval, parsing, documentation and governance.

Two findings from the first run were discarded before publication, and an instrument that audits other people should say which. The first attributed a transport failure to the wrong party: the DCAT-US SHACL shapes and the National Archives restrictions vocabulary were retrieved from raw.githubusercontent.com, which serves Turtle as text/plain, and the register booked a media type violation against GSA and NARA for a header GitHub sets. Those assets are now flagged and transport checks return inapplicable with the reason stated. The second was a check measuring its own regular expression: it derived namespace prefixes by truncating term URIs and reported http://id.loc.gov/vocabulary/ as a dead dependency of twelve assets, a URL that does 404 but that no publisher ever minted. The check now probes the referenced terms themselves over a deterministic sample, and the corrected count is six assets rather than eighteen.

A third case changed shape rather than disappearing. The NOAA paleoenvironmental thesaurus first recorded as unparseable. It is served as application/rdf+xml; charset=ISO-8859-1, carries no XML declaration, and contains a byte that is not valid UTF-8 at offset 9,497, the n-with-tilde in "El Nino". A correct HTTP client honours the charset parameter, so the loader now does and the file parses to 27,326 triples. The real issue is different from a parse failure and is now its own check: detached from its HTTP headers, the file cannot be decoded. XML 1.0 expressly permits an external protocol to supply the encoding, so it is graded conventional.