UK health data linkage registers, measured at the discovery layer

Why now

The Sudlow Review, published on 8 November 2024 and commissioned by the Chief Medical Officer for England, the UK National Statistician and NHS England's National Director for Transformation, made the case that England's health data should be treated as critical national infrastructure, and its premise is linkage: the value of the country's health data comes from joining its sources together. On 7 April 2025 the government and Wellcome announced up to 600 million pounds, 500 million from government and 100 million from Wellcome, to build a Health Data Research Service at the Wellcome Genome Campus. The institutions that will build and staff that service are hiring data linkage and data architecture people now. Wellcome itself opened a Technology Lead vacancy for data linkage and data architecture in its Discovery team on 26 August 2026.

So the money, the mandate and the hiring all assume that linkage between UK health datasets can be described in a way machines can follow. That assumption is measurable today, on public registers, with no credentials. This study measures it.

What we measured, and from where

Two public surfaces, both open and keyless, both pinned to 28 August 2026. The HDR UK Gateway API at api.healthdatagateway.org: 1,707 dataset records (1,278 active, 121 draft, 308 archived, every one converting to Gateway Data Model 2.0), 2,565 data use register rows, 3,459 publication records, 427 tools and 218 collections. And the NHS England Data Uses Register, published monthly as a workbook: 5,577 data sharing agreement versions, 38,650 agreement-to-dataset rows, and 103,706 released files, plus the internal register of 31 agreements and 428 data sharing framework contracts.

The conformance targets are not typed by hand. The Gateway Data Model and HDRUK schemas are fetched from HDR UK's own schemata repository, which is public under Apache 2.0, and each of the 1,278 active records is validated against the GWDM 2.0 JSON Schema exactly as published. A linkage target counts as resolved if its identifier matches a Gateway pid, its URL points at a harvested record, or its title matches a harvested dataset title exactly. That rule is implemented twice, once over Python sets and once by SPARQL over an RDF graph of 88,772 triples, and the build fails if the two computations ever disagree. On this run, seven checks out of seven agree.

Finding one: the dataset-to-data-use link is dead metadata

The Gateway Data Model defines linkage.dataUses on every dataset, the field that says which approved data uses draw on this dataset. On 28 August 2026 it is empty on all 1,278 active datasets. Not sparse. Empty.

This is not because the information does not exist. The Gateway runs a data use register with 2,565 rows, and 1,704 of them are relationally linked to Gateway datasets through the platform's own database. The system knows which uses draw on which datasets. The metadata field the standard defines for exactly that relationship carries none of it, so anyone consuming Gateway metadata through the standard, rather than through the platform's internal joins, sees a register in which no dataset has ever been used.

The operational consequence lands on anyone building transparency or impact reporting on top of the metadata: counting the uses of a dataset from GWDM output returns zero for every dataset in the country.

Finding two: a third of declared dataset links resolve to nothing

109 active datasets declare links to other datasets, 324 entries in all. Each entry is a descriptor with three slots: pid, url and title. 223 of the 324 resolve to a Gateway record through at least one slot. The other 101 resolve to nothing:

What sits in the title slot is often not a title. 81 of the 288 published titles run past 80 characters, and the register's most repeated linkage entry reads, verbatim: "Each patient will be identified by an unique patient key that can be used to link to all other datasets available within London SDE Platform". That is a true and useful sentence for a human. It is also the entire machine readable content of the linkage: no identifier, no URL, no target.

The distribution across custodians matters, because it shows the problem is practice, not possibility. London Secure Data Environment accounts for 41 unresolvable entries and North West SDE for 23, with Generations Study at 10, Genomics England at 8 and Public Health Scotland at 7. Meanwhile University of Sheffield's Data Connect publishes 169 linkage entries and Thames Valley and Surrey SDE 41, almost all resolving by Gateway URL or exact title. Two custodians prove the same schema, on the same platform, supports linkage a pipeline can follow end to end.

Finding three: the identifiers are names

59 linkage entries populate the pid slot, the slot for a persistent identifier. Every one of the 59 values is a dataset name: "Electronic Patient Record - OUH NHS FT", "Cancer Waiting Times (CWT) - OUH NHS FT", "Radiology reports - OUH NHS FT". Zero are UUID shaped. We checked them against every generation of Gateway identifier, current pids, the pre-migration Mongo identifiers and the numeric dataset ids, and zero resolve.

The mechanism is a schema decision. In HDR UK's published schema, a dataset's own identifier is typed Uuidv4, but the pid slot inside a linkage descriptor is typed as a general string of 2 to 150 characters. Prose in the pid slot therefore validates, and publishers have used it as a label field. The one character class of fix, typing linkage pids as identifiers, would make this entire failure mode impossible at submission time.

The register's own identifiers have a quieter version of the same problem. 133 of the 1,707 dataset pids, 131 of them on active records, are UUID shaped but fail the Uuidv4 type the schema declares: wrong version and variant bits, with sequential tails such as …9ce46, …9ce47, which is what a migration script minting pseudo-UUIDs looks like. Nothing checks, so nothing complains.

Finding four: the two registers barely share an identifier

The UK publishes two registers of health data uses. NHS England's Data Uses Register records data sharing agreements under DARS-NIC references, 5,577 agreement versions in 1,942 families in the August 2026 export. The Gateway's data use register holds 2,565 rows with a free text project id field. Joining them by identifier is possible for 68 rows. That is the overlap: 65 agreement families, 59 of which appear in the NHS register, out of 1,942. 96.96 per cent of the NHS register's agreement families have no identifier-joinable entry on the Gateway.

The reverse direction is small but sharper. Six DARS families on the Gateway are absent from the NHS external register; two of them appear in the internal register, which leaves four, all approved in 2021, all COVID era projects, shown as active data uses on the Gateway while absent from both current NHS England publications. Two public registers disagree about whether these uses of national health data are current, and neither says so, because neither reads the other.

Datasets fare no better than agreements. The NHS register references datasets by free text name, 211 distinct names across 38,650 rows. Twelve of the 211 match a Gateway dataset title exactly. The country's record of which datasets were released under which agreements, and its catalogue of what those datasets are, connect by name in twelve cases and by identifier in none.

What this is not

This is not a data quality scandal, and it is worth being precise about why. The NHS England register is internally disciplined to a degree worth stating: all 5,577 agreement references conform to the DARS-NIC pattern, there are zero duplicates, and across 38,650 dataset rows and 103,706 release rows there is not one dangling cross-sheet reference. Its residual defects are seven agreements with no dataset rows and one agreement whose end date precedes its start date. HDR UK, for its part, publishes its schemas in the open under Apache 2.0, computes a metadata quality score per dataset, and published the data utility framework behind it (Gordon and colleagues, BMJ Health and Care Informatics, 2021). GUILD (Gilbert and colleagues, Journal of Public Health, 2018) set out years ago what researchers need to know about linkage processing. None of that ground is claimed here.

What this study adds is the layer none of those instruments look at: whether linkage claims resolve by machine, and whether the registers agree with each other. 1,277 of 1,278 active Gateway records validate in full against GWDM 2.0. Every empty descriptor, every prose pid and every dangling target above passes schema validation, because a JSON Schema cannot express "this reference must resolve" or "these two registers must agree". Those are graph properties, and they need graph checks.

The model: linkage as a claim, not a property

The Health Data Linkage Ontology (HDLO) is a small OWL 2 ontology with one load-bearing decision: linkage is never a property of a dataset. It is a dated, reified claim by a named register. ALinkageAssertion carries whatever the register actually published about the target, pid, URL, title or nothing, verbatim, plus a computed machineActionableflag. An IdentifierAssertion records a published identifier against the conformance rules its scheme declares as data in a SKOS registry, so the pipeline validates against the scheme rather than against hard coded rules, and a test pins the SHACL shapes to the registry so the two cannot drift. A CrossRegisterObservation records what two registers say about the same entity, including the case where one is silent. Silence is a position.

Three SHACL layers run over the emitted graph: structural integrity, scheme conformance, and one shape per defect class, so the validation report is the findings table. On this run: layer one conforms, layer two reports exactly the 133 pseudo-UUID pids, and layer three reports 224 machine-unactionable linkage assertions, 29 dangling pointers, 63 prose titles and 1,883 cross-register silences, reconciling with the dual-computed counts.

What would move the numbers

For HDR UK, two schema changes and one data fix. Type the pid slot of a linkage descriptor as an identifier rather than a 150 character string. Constrain descriptors so an entry with no pid, no URL and no title cannot validate. And repair or re-mint the 133 pseudo-UUID pids. We are filing the schema findings, including a separate defect in the DOI pattern that accepts strings like 10X1234/abcd, as issues on the schemata repository.

For custodians, the Sheffield comparison is the whole prescription: publish linkage entries with the Gateway URL or exact title of the target, which two custodians already do at scale on the same platform. For the register pair, one shared reference would do more than any metadata programme: a DARS-NIC field on Gateway data use entries, populated at approval time, would take the identifier join from 3 per cent to wherever NHS England's coverage of the register actually stands, and would surface disagreements like the four ghost agreements automatically.

For the Health Data Research Service, the finding is a baseline: the discovery layer it inherits asserts linkage mostly in prose. Requiring resolvable identifiers in linkage metadata, at submission time, is cheaper than reconstructing linkage after the fact, and the registers this study measured are the systems it will federate.

Method, reproducibility, and the offer

Everything is public and reproducible from two open sources with no credentials. The repository at github.com/fabio-rovai/health-data-linkage-ontology holds the resumable harvest pipeline, the ontology, the SKOS scheme registry, the SHACL layers, the verified SPARQL queries, the tests, and a build report that states populations, methods, the errors made during the build, and the claims that died on verification. One did: an early version of finding three read as a resolution failure until we looked at the values and found names, which is a different and stronger finding.

The method transfers to any register pair: harvest both sides resumably, declare each identifier scheme's rules as data, reify every linkage claim with what was actually published, compute actionability, and compare registers with silence recorded as a position. We have run the same playbook on EU health dataset catalogues, fund registers, bank registers and the scholarly record.

If your organisation publishes to the Gateway, runs a secure data environment, or is building on the Health Data Research Service and wants its linkage layer to resolve, the first engagement is a fixed scope review of your register's linkage metadata against these shapes, with a findings table your data architecture team can act on. Write to fabio@thetesseractacademy.com.