Foundational Research: Ontology-Native AI
SHACL validates shapes, not vocabulary
A companion to our benchmark, for the people who build with SHACL. The 100% false-pass rate we measured is not a defect in any validator. It is the open-world semantics of SHACL Core doing exactly what the specification says. Understanding why is the difference between trusting SHACL for something it was never meant to do and putting the right check in the right place.
What SHACL was designed to do
SHACL, the W3C Shapes Constraint Language edited by Holger Knublauch and Dimitris Kontokostas, validates an RDF data graph against a shapes graph. A shape declares a target (a class, a specific node, the subjects of a property) and a set of constraints on those targets: cardinality, datatype, value range, node kind. The validator visits the targeted nodes, checks the declared constraints, and reports the violations. This is a precise and useful contract. It is also, deliberately, a local one. SHACL never asserts that the shapes graph is complete, and it never claims authority over triples no shape mentions. That is the open-world assumption, and it is a feature: RDF graphs are meant to be mergeable and extensible, so a validator that failed every graph carrying an unforeseen predicate would be useless in practice.
Why a fabricated term is invisible
Now feed that validator a graph an LLM wrote. It types a node as a real class, satisfies the real constraints, and adds a predicate that does not exist: schema:priceBracket where the vocabulary only defines schema:priceRange. No shape targets priceBracket, because you cannot write a shape for a term you did not know would be invented. So SHACL never looks at it, and a graph with no violations is a conformant graph. The fabricated term is not passed despite scrutiny; it is passed because it was never in scope. Across 300 hallucinated graphs on three real vocabularies, that happened every single time.
"But sh:closed"
SHACL does provide sh:closed, which flags any property on a targeted node that is not in an explicitly permitted list. It is the nearest thing in the language to a vocabulary check, and it genuinely helps. But it is not the guarantee it looks like. Closing a shape means enumerating, by hand, every property that node is ever allowed to carry, and keeping that list current as the model evolves. It operates per shape, so a node with no shape is still unchecked. It does not constrain the values of rdf:type against the vocabulary, so a fabricated class slips through even on a closed shape. And it breaks the moment a record legitimately uses a term from an imported ontology you did not list, which for anything built on the OBO Foundry (PATO imports RO imports BFO) is the normal case, not the exception. sh:closed is per-shape bookkeeping that trades away the extensibility SHACL was designed to protect. It is not a statement about the vocabulary.
The right check in the right place
The question SHACL answers is "do the terms I was told to check satisfy their constraints?". The question generated data forces is a different one: "does every term in this graph denote something the ontology actually defines?". That is a closed-world membership test, and it belongs beside SHACL, not inside it. Run it as its own pass: collect every predicate and every rdf:type value, and flag any whose IRI sits in a namespace the ontology owns but that the ontology never declares. It leaves SHACL to do the structural job it is good at, and it leaves the extensibility intact, because it polices only the namespaces you name and never the standard rdf, rdfs, owl vocabulary or your own instance IRIs. On 300 clean graphs it raised zero false alarms. This is the onto_vocab_check tool in our open-ontologies engine: not a replacement for SHACL, a companion constraint SHACL Core cannot express.
"SHACL is not failing when it passes a hallucinated term. It is answering the question it was asked. The mistake is asking it the wrong question and calling the answer trust."
Fabio Rovai, Tesseract Academy
The full method, the three vocabularies, and every number are in the benchmark write-up and reproducible from the repository. If you build with SHACL and you also let a model author RDF, this is the gap you are carrying.
The closed-world companion
Reproduce the benchmark, or run the vocabulary gate on your own ontology.
