Case Study: AI for Government Data Standards
Teaching an open-source LLM a government information standard: from 94% hallucination to 1%
We fine-tuned Qwen3-Coder-30B, an Apache-2.0 licensed 30-billion-parameter mixture-of-experts model, on IES4, the UK government's Information Exchange Standard used across defence and national security data sharing. The result, published openly on Hugging Face, shows what it takes to make a language model respect a controlled vocabulary: measurement, not assertion.
Term confabulation
93.7% → 1.0%
invented standard terms, near-eliminated
Term conformance
0 → 88.6%
outputs conforming to the IES4 vocabulary
Published
Openly
model released on Hugging Face
The Challenge
Regulators and public bodies increasingly want AI that works with their structured standards: controlled vocabularies, coded classifications and formal ontologies. The problem is that general-purpose language models do not respect them. Asked to produce data conformant to IES4, the UK government's Information Exchange Standard for defence and national security data sharing, an untuned open-source model invented standard terms that do not exist 93.7% of the time. In a standards context that failure mode is dangerous precisely because it looks plausible: fabricated terms sit alongside real ones and pass a casual read.
The question we set out to answer was practical. Can an open-source model be taught a specific government information standard well enough to be useful, and can the improvement be proven with machine checks rather than claimed on trust?
What We Delivered
Kampakis and Co, trading as Tesseract Academy, fine-tuned Qwen3-Coder-30B on IES4 and published the resulting model openly on Hugging Face. The distinctive part of the work is not the fine-tune itself but the verification discipline around it: every quality claim is backed by a deterministic, machine-run check against the published standard.
| Principle | How it works |
|---|---|
| Closed-world vocabulary verification | Every term the model generates is machine-checked against the actual vocabulary of the published IES4 ontology. A term either exists in the standard or it does not; there is no room for interpretation. |
| Machine-verifiable ground truth | The evaluation harness is built directly from the structured data of the standard itself, so the quality numbers are computed by deterministic checks, not scored by another language model or by human impression. |
| Open publication | The fine-tuned model is published openly on Hugging Face, so any public body can inspect it, reproduce the evaluation and hold the claims to account. |
The Results
Both numbers below are computed by the closed-world vocabulary verifier, which checks every generated term against the standard's actual vocabulary. They are measured, not estimated.
| Metric | Base model | Fine-tuned | Meaning |
|---|---|---|---|
| Term confabulation | 93.7% | 1.0% | generated standard terms that do not exist in IES4 |
| IES4 term conformance | 0% | 88.6% | outputs that use only real terms from the published standard |
A fuller technical account, including the training-data construction and further conformance metrics, is in our research write-up: an open language model for IES4 data.
Why This Matters for the Public Sector
Public bodies do not need AI that sounds fluent about their standards; they need AI whose outputs conform to them, and evidence that it does. The general capability demonstrated here is building evaluation harnesses with machine-verifiable ground truth derived from structured data: wherever an organisation has a controlled vocabulary, a coded classification or a formal schema, the same method turns "the model seems good" into a conformance number that anyone can recompute. That is the difference between quality that is asserted and quality that is engineered.
"If a standard is machine-readable, then conformance to it is machine-checkable, and there is no excuse for taking an AI system's quality on faith. We measure it, we engineer it, and we publish the result so others can check our work."
Fabio Rovai, Tesseract Academy
The model, published openly
Inspect the fine-tuned model on Hugging Face, or read the full technical write-up.
