Safe AI

The failure that actually happens

The AI failure that reaches a public body is rarely dramatic. It is a fluent system producing a confident answer that is wrong, in a pipeline where no component is capable of noticing. Retrieval does not notice, because retrieval returns relevant text and relevance is not truth. The model does not notice, because the same machinery generates the error and the explanation of it. A human reviewer does not notice at volume, because the output reads exactly like the outputs that were right. Safety, in the only sense that survives contact with an audit, means putting something in the loop that is able to fail.

There are three places to put it, and we have published a measured reference implementation for each. The first is the term level: check what the model asserts against a published vocabulary it does not get to extend. The second is the action level: permit an action only if it carries a certificate a small trusted component can verify. The third is the system level: generate the governance evidence from the running system rather than writing a document about it.

• The gate most teams think they already have does not exist. Open-world SHACL accepted all 300 data graphs we seeded with a fabricated term. A closed-world vocabulary gate caught all 300, with no false positives on clean data. If a language model is writing your RDF, shape validation is not the check you think it is.
• Grounding a model in a standard moves the numbers. On the UK Information Exchange Standard, confabulated terms fell from 93.7 per cent of outputs to 1.0 per cent and conformance rose from zero to 88.6 per cent. On a space object catalogue, hallucinated terms fell from 13.81 per output to 0.06.
• An action gate can be verified rather than tested. Our reference proof-carrying-action gatekeeper blocks 672 of 672 unsafe actions and admits 504 of 504 safe ones across 96 reachable states and 1,176 transitions, at 0.36 microseconds per check with a 31-line trusted core. Remove it and 57 per cent of actions violate the safety specification.
• Auditing one safety property at a time hides the trade. On real clinical readmission data, tightening differential privacy cost minority subgroups 2.6 times more accuracy than the majority while the membership leakage it targeted was already near zero. Almost all the equity cost, almost none of the privacy gain, visible only when both planes sit on one report card.

The regulatory layer sits on top of this and does not replace it. The EU AI Act, the NIST AI Risk Management Framework and ISO/IEC 42001 all require documented risk classification, monitoring and oversight, and none of them specifies what a sufficient technical check looks like. That is the gap most programmes fail an audit in: the policy exists and the evidence does not. Our Open Governance server is the open-source answer to that half, discovering AI systems, classifying risk against all three frameworks and emitting audit-ready compliance matrices through 48 governance tools. The AI governance service is how it gets applied.

Why the check is usually an ontology

An ontology is a formal, machine-readable model of a domain: what kinds of thing exist, what properties they have, what relationships can hold between them, and what constraints must hold. It is normally written in OWL 2, with SKOS carrying the controlled vocabulary and SHACL carrying the constraints. Writing it formally buys you one thing that a data dictionary or a Confluence page cannot: software can read it and reject a statement that contradicts it.

That capability is the whole reason ontologies came back, and it is what makes them a safety component rather than a documentation exercise. An ontology is the cheapest artefact that can tell a system it is wrong at the term level, without a human in the loop and without a labelled answer key to compare against. Everything above depends on having one: you cannot run a closed-world check against a vocabulary nobody wrote down, and you cannot issue a certificate about an action whose preconditions were never modelled formally.

The distinction people most often get wrong is between the three layers. A taxonomy is a hierarchy of concepts. An ontology adds the relationships and the constraints. A knowledge graph is the instance data underneath: the actual entities and the actual claims about them. The ontology says what can be said. The graph says what is being claimed. Our glossary covers the rest of the vocabulary, and the free 15-lesson ontology engineering course teaches the modelling itself, from RDF through SPARQL, SHACL and GraphRAG.

Questions we get asked

What does safe AI mean in practice, for a public body?

It means the system cannot take an action, or serve an answer, that nothing was able to check. Most deployed AI is unsafe in a boring way rather than a dramatic one: it is fluent, it is wrong, and no component in the pipeline is capable of noticing. Making it safe means adding something that can fail. That can be a closed-world vocabulary gate over what the model asserts, a certificate a small trusted component verifies before an action is permitted, or a measured report card that puts privacy and fairness on the same page instead of auditing them separately. All three are engineering, all three are cheap next to the incident, and all three produce evidence an auditor can read.

Why is guardrail prompting not enough?

A guardrail written in the prompt is checked by the same system that is being guarded, so it fails in exactly the cases where you needed it. A gate is different: it sits outside the model, it is small enough to verify, and it says no. We built a reference proof-carrying-action gatekeeper for a bounded multi-agent system and verified it exhaustively rather than testing it: across 96 reachable states and 1,176 transitions it blocked 672 of 672 unsafe actions and admitted 504 of 504 safe ones, at 0.36 microseconds per check, with a trusted core of 31 lines. Without the gate, 57 per cent of actions violated the safety specification.

How do you stop a language model inventing things?

You cannot stop it, so you catch it. The cheapest place to catch it is at the term level, against a published vocabulary the model does not get to extend. When we fine-tuned an open model on the UK Information Exchange Standard, confabulated terms fell from 93.7 per cent of outputs to 1.0 per cent and conformance rose from zero to 88.6 per cent. The same method on a space object catalogue took hallucinated terms from 13.81 per output to 0.06. The number that matters more than either is the check itself: without a closed-world gate, fluent and wrong is indistinguishable from fluent and right.

Does the EU AI Act or ISO 42001 require this?

They require you to be able to show it. The EU AI Act, the NIST AI Risk Management Framework and ISO/IEC 42001 all demand documented risk classification, monitoring and human oversight, and none of them tells you what a sufficient technical check looks like. That gap is where most programmes fail an audit: the policy exists, the evidence does not. We publish Open Governance, an open-source governance server that discovers AI systems, classifies risk against those three frameworks and produces audit-ready compliance matrices, so the paperwork is generated from the running system rather than written about it. One timing correction worth having: the Digital Omnibus moved the Act’s Annex III high-risk obligations from 2 August 2026 to 2 December 2027, but the Article 50 transparency duties, which cover chatbot disclosure and synthetic-content marking, took effect on 2 August 2026 and are already live.

What is an ontology?

An ontology is a formal, machine-readable model of a domain: the kinds of thing that exist in it, the properties those things have, the relationships that can hold between them, and the constraints that must hold. In practice it is written in OWL 2, usually with SKOS for the controlled vocabulary and SHACL for the constraints. The point of writing it formally is that software can then reason over it and reject statements that contradict it, which prose documentation cannot do.

What is the difference between an ontology, a taxonomy and a knowledge graph?

A taxonomy arranges concepts in a hierarchy: broader and narrower, and little else. An ontology adds the relationships and the constraints, so it can express that a bank holding company controls a subsidiary and that control has to be dated and sourced. A knowledge graph is the instance layer: the actual banks, the actual subsidiaries, the actual dates. The ontology says what can be said; the knowledge graph says what is claimed. You can build a graph without an ontology, but then nothing can tell you the graph is wrong.

What is the difference between OWL and SHACL, and do I need both?

OWL 2 describes what is true in the domain and lets a reasoner infer more of it. SHACL checks whether a particular data graph conforms to a set of shapes. They answer different questions, so most real projects use both. The trap is assuming SHACL is a sufficient check. SHACL is open-world: if your data uses a property the shapes say nothing about, SHACL passes it. We measured this across three vocabularies and open-world SHACL accepted every one of 300 graphs carrying a fabricated term. If a language model is writing your RDF, you need a closed-world vocabulary gate as well.

Is the semantic web the same thing as a knowledge graph?

No, though the technologies overlap almost completely. The semantic web is the W3C programme and its stack: RDF for the data model, RDFS and OWL for the schema, SKOS for vocabularies, SHACL for validation, SPARQL for querying, and dereferenceable URIs so that data published by different people can be joined. Linked data is the publishing discipline that goes with it. A knowledge graph is the artefact you get: a graph of entities and relationships, usually stored in a triple store, that a system actually queries. Most enterprise knowledge graphs are built on the semantic web stack precisely because they need identifiers that survive being joined against somebody else’s data.

What is GraphRAG, and does it need an ontology?

GraphRAG retrieves over a knowledge graph rather than over a flat vector index, so the model gets structured neighbours and relationship paths instead of loose passages. It works better than plain retrieval on multi-hop questions. It does not need an ontology to run, and this is where projects go wrong: without a schema and a term-level check, the graph the pipeline builds inherits every entity the extraction step invented, and GraphRAG then retrieves that error confidently. The ontology is what lets you reject the bad node before it enters the graph.

Do AI agents and RAG pipelines actually need an ontology?

They need something that can tell them they are wrong, and an ontology is the cheapest thing that does that job at the term level. Retrieval gives a model relevant text; it does not give it a way to detect that the entity it just asserted does not exist. When we fine-tuned an open model on the UK IES4 standard, confabulated ontology terms fell from 93.7 per cent of outputs to 1.0 per cent, but the number that made that measurable at all was the closed-world check against the published vocabulary. Without it, fluent and wrong looks the same as fluent and right.

How long does an ontology project take, and what does it cost?

A scoped audit of an ontology or knowledge graph you already have takes about two weeks and sits below the £10,000 direct award threshold for public bodies. A domain ontology with a validated instance graph over real source data is typically six to twelve weeks depending on how many source registers have to be reconciled. Public sector buyers can commission through CCS RM6200 for build work or RM6126 for research and audit work.

Who builds ontologies in the UK?

The established UK and European names include Semantic Partners, Ontotext, DNV and the Ontology Engineering Group at Universidad Politécnica de Madrid, alongside in-house teams at large publishers and banks. Tesseract Academy works in the same space with a specific emphasis: everything we publish is reproducible from public data, every headline is computed at least two independent ways, and we report the hypotheses that died. Our work for the National Digital Twin Programme is open source under Apache 2.0.