Foundational Research: Ontology-Native AI
An open language model for IES4 data
To our knowledge the first openly published language model fine-tuned for the UK Information Exchange Standard (IES), the 4D ontology behind defence and security data. It turns a plain-English scenario into IES-conformant RDF, and it is released as a research prototype with its dataset and evaluation harness so the field can check the work.
Term conformance
0 → 88.6%
vs the untuned base model
Hallucinated terms
0.937 → 0.010
near-eliminated
Released
Model + data
and full eval harness
The Challenge
The Information Exchange Standard (IES) is the shared 4D vocabulary for UK defence and security data, but authoring IES-conformant RDF by hand is slow and needs scarce ontology expertise. General-purpose models do not fill that gap. Asked to write IES Turtle, even a strong 30B code model invents terms that do not exist in the ontology roughly 94% of the time. In a standards context that is worse than no help at all: invalid data that looks plausible is harder to catch than data that fails outright.
What we built
The core idea is to never trust the model to invent structure. Instead we generate the structure correctly, then teach the model to reproduce it. Every training graph was built programmatically with the telicent-ies-tool and validated twice before training.
| Component | What it is |
|---|---|
| Correct-by-construction data | 1,589 IES graphs built programmatically with telicent-ies-tool across 14 scenario patterns, paired with natural-language descriptions. |
| Double validation | Every graph passed both the telicent validation and an independent term-membership check built from the published dstl/IES4 ontology. |
| Vocabulary and boundaries | Class and property definitions from the ontology, plus refusal examples teaching what IES4 cannot express. |
| Generalisation slice | Ontology-conditioned extraction pairs from Text2KGBench, so the model conforms to any ontology it is given, not only IES. |
| Honest evaluation | Held-out test split by target graph, plus an out-of-distribution set from the real published IES sample data. |
| Open release | Model, LoRA adapter, full dataset and evaluation harness, all published for independent reproduction. |
The results
Measured on held-out prompts against the untuned base model. The point of the fine-tune is not fluency, which the base model already has, but conformance: using real IES terms, in structurally valid ways.
| Metric | Base model | Fine-tuned | Meaning |
|---|---|---|---|
| IES4 term conformance | 0% | 88.6% | terms used that exist in the ontology |
| Hallucinated-term rate | 0.937 | 0.010 | fraction of ies: terms that are invented |
| Syntactic validity | 93.2% | 95.5% | output parses as valid Turtle |
| Structural conformance | n/a | 0.955 | subject and object types satisfy domain and range |
| Namespace fidelity | n/a | 100% | uses the namespace the caller specifies |
| Ontology-conditioned extraction | 75% | 91.7% | conforms to a supplied non-IES ontology |
Out-of-distribution performance, on rich scenarios taken from the real published IES sample data rather than our training patterns, is markedly lower (term conformance around 30%). We report that openly in the model card rather than hide it: it is the honest edge of what a small on-device fine-tune achieves today, and the reason the model is released as a prototype to draft from, not an oracle to trust blind.
Does the model earn its place? A neuro-symbolic test
The obvious challenge to a fine-tuned model is: why not skip it, and instead wrap a general model in a validate-and-repair loop? Generate the Turtle, check it against the ontology, feed the specific errors back, and ask the model to fix them. We ran that experiment directly. Two models (the untuned base and this fine-tune), each on its own and inside a three-round repair loop, on both the in-distribution and the harder out-of-distribution test sets. The number below is IES4 term conformance.
| Setup | In-distribution | Out-of-distribution |
|---|---|---|
| Untuned base, single-shot | 0% | 0% |
| Untuned base, with repair loop | 0% | 0% |
| Fine-tuned, single-shot | 95% | 30% |
| Fine-tuned, with repair loop | 100% | 50% |
The result is clean, and it refutes two tempting shortcuts at once. First, the repair loop cannot replace the fine-tune: on the untuned base it changes nothing (0% stays 0%), because a model that does not know the vocabulary cannot act on the feedback "that term does not exist", it simply invents a different wrong one. Second, the failure mode of the fine-tuned model is coverage, not capacity: its lower out-of-distribution score reflects IES patterns it has not seen, which a larger model would not fix but broader training data would. What the loop does earn is real: on a model that already holds the knowledge it lifts in-distribution conformance to a perfect 100% and pushes the hard out-of-distribution case from 30% to 50%, halving the hallucination rate along the way.
So the fine-tune earns its place as the load-bearing component, and the validator earns its place as the layer that cleans up on top. Neither is sufficient alone.
The last mile: what actually breaks the out-of-distribution wall
We then tested the obvious fixes, one variable at a time, and the result overturned our own first guess. Out-of-distribution term conformance turned out to be invariant to data volume (a 10x larger training set) and to linguistic diversity: it stayed pinned at 30%. A later model (v1.3) that added the exact missing IES construct families lifted in-distribution conformance to 97.7% and improved out-of-distribution structure by 40%, yet term conformance still held at 30%. Diagnosing every residual failure showed why: the model was not missing constructs, it was inventing plausible-but-wrong term spellings on unfamiliar scenarios (writing ies:hasParticipant for ies:isParticipantIn, or ies:MeasureUnit for ies:measureUnit).
A plain repair loop that only says "that term is invalid" lifts this from 27% to 45%: it fixes the cases where the model can guess the correction, and stalls where it cannot. A term-suggesting validator that returns the nearest real IES term ("use ies:isParticipantIn") closes most of the rest.
| Pipeline (v1.3, out-of-distribution) | IES4 term conformance |
|---|---|
| single forward pass | 27% |
| + repair loop ("term is invalid") | 45% |
| + term-suggesting validator ("use X") | 82% |
The lesson is precise and, we think, general: for niche-format generation the binding constraint is not scale or diversity but term-exactness on novel compositions, and the fix is a validator that supplies the correct term, not merely flags the wrong one. The fine-tune drafts; the symbolic layer perfects. This is a small-scale result (tens of held-out prompts, one ontology), reported as a finding to build on.
Where it fits
The right architecture for standards work is neuro-symbolic: the model drafts, and a symbolic layer verifies. This model pairs directly with our IES-to-HQDM crosswalk and the open-ontologies validation engine, which check conformance deterministically. The language model removes the blank-page problem; the validators guarantee correctness. Neither is asked to do the other's job.
"A general model asked to write a niche standard will confidently make it up. The fix is not a bigger model, it is correct-by-construction data and a validator in the loop. Publish both, with the weak spots named, and the field gets a tool it can actually build on."
Fabio Rovai, Tesseract Academy
This is an independent, self-initiated demonstration built entirely on published open standards and open tools. It is not affiliated with, or endorsed by, the Ministry of Defence, the IES Working Group, Telicent or Dstl. IES4 is Crown Copyright, MIT-licensed; the model weights are MIT; the generalisation data derived from Text2KGBench is attributed under CC-BY-SA-4.0 in the model card.
Model, dataset and evaluation harness
Weights and LoRA adapter, the full training data, and the code that produced every number above.
