Case Study: Space Environment Measurement
Space debris metrics that cannot be added up
The world's two premier orbital debris models disagree by a factor of 2.3 to 3.0 at the diameter where collision-avoidance decisions are made. Neither is wrong. They divide the same physical population along axes that do not align, and nothing in either model's output records that fact. We built an open checker that catches it.
Divergence at 1 cm
2.3 to 3.0x
between two authoritative models
Standards used
3
QUDT, SOSA/SSN, PROV-O
Open and tested
9 / 9
tests passing, MIT licensed
The Challenge
ESA's MASTER-8 and NASA's ORDEM 3.1 are the two premier orbital debris engineering models. Both are authoritative. Both are carefully built. Both describe the same physical reality. On identical simulation cases they diverge sharply, and the joint ESA and NASA comparison study is explicit that this is not a matter of one being right: "each model provides its agency's best estimate of the orbital debris environment. One model may be more accurate in one regime while the other may be more accurate in another, and it is expected that the truth lies somewhere in between."
The structural cause is stated in the same paper, and it is the sentence that matters: "a direct comparison of the source models is not immediately possible from the model output." MASTER-8 divides the population by generative source, into fragments, solid rocket motor slag and dust, NaK droplets, paint flakes, ejecta and MLI fragments. ORDEM 3.1 divides it by material density, into low, medium, high, NaK and intacts. One population, orthogonal cuts, only the NaK component in common.
Near identical
Less than an order of magnitude, ORDEM 3.1 lower
MASTER-8 higher by a factor of 2.3, and 3.0 in the second case
ORDEM 3.1 higher by a factor of 2 to 10
ORDEM 3.1 almost two orders of magnitude higher
The 1 cm row is the one that matters operationally. The comparison paper calls it "a critical diameter threshold for collision risk estimations", and it is where collision-avoidance thresholds, shielding specification and post-mission disposal requirements are set.
Why this is a safety problem, not a housekeeping problem
Every environmental indicator is computed over an underlying population model, and therefore silently inherits that model's partition. ECOB, the Criticality of Spacecraft Index and the Space Sustainability Rating mission index each carry an inheritance their outputs do not record. Combine two indicators built on different models and you have combined two incompatible partitions with no trace that it happened.
There is no error message. The composed indicator returns a number, and the number gets used. It is the same class of failure as a unit mismatch, and it has the same property: cheap to prevent by construction, effectively undetectable afterwards.
What we built
Each metric is declared as an explicit, machine-readable object rather than described in prose. We deliberately built on mature standardised semantic infrastructure rather than inventing anything: QUDT for quantity kinds and units, SOSA/SSN (a W3C Recommendation) for observation semantics, and PROV-O (a W3C Recommendation) for the assumption and derivation chain. The vocabulary adds only the two things those three do not supply: the population partition a metric inherits from its source model, and its regime of validity.
DimensionQuantity kinds must match for additive composition. Carried by QUDT.
DenominatorQuantities normalised against different bases are not additive. Per mission and per resident space object are not interchangeable.
PartitionSource models that partition the same population along orthogonal axes cannot have their outputs arithmetically combined. This is the rule the domain currently lacks.
Regime of validityDeclared size ranges and orbital regimes must overlap, and any requested evaluation point must sit inside the intersection.
IndependenceMetrics sharing a source model are flagged as non-independent: common-mode error means a composed figure understates uncertainty.
The decision is symbolic. There is no statistical component and no language model anywhere in the decision path, so the same inputs always produce the same, inspectable answer.
Four verdicts, not one refusal
| Composition | Verdict | Why |
|---|---|---|
| ECOB + ORDEM-derived indicator | INVALID | R3, orthogonal partitions |
| ECOB + Criticality of Spacecraft Index | INVALID | R2, per mission against per object |
| ECOB + SSR Mission Index | VALID WITH CAVEAT | R5, both derive from MASTER-8, so not independent |
| Any pair, reported side by side | VALID | No arithmetic performed |
The third row is the one that shows the tool is doing real work. A checker that refuses everything is useless, and one that permits everything is dangerous. ECOB and the SSR mission index genuinely are structurally combinable, and the framework permits the combination while flagging that both derive from MASTER-8 and are therefore not independent estimates. Shared source-population error is common-mode, so a composed figure will understate its own uncertainty. That is a judgement a spreadsheet cannot make and a dashboard will not surface.
Every refusal is issued together with what may be done instead: report both figures side by side attributed to their source models with the divergence stated; use either metric alone within its own declared regime of validity; or establish an explicit, evidenced mapping between the two partitions, after which composition becomes checkable. The purpose is to establish what can validly be done. Refusal is the safety property underneath, not the headline feature.
The gap this sits in
A decade of orbital debris ontology work exists in the literature. The Space Object Ontology, the Space Situational Awareness Ontology and an associated orbital event ontology are all formalised in OWL, and all are described by their authors as under development. None has been mapped to an operational data standard or to an engineering model.
Meanwhile the CCSDS orbit, conjunction and tracking message families are mature and operationally adopted, but they are syntactic: they standardise message structure, not the meaning of a derived quantity, not the population partition it was computed over, and not the assumptions under which it holds. Four layers, developed independently, none mapped to the next. This artifact is one small, concrete step across that gap.
Scope, stated plainly
The declarations capture the structural commitments of each metric: quantity kind, denominator, source model, inherited partition, regime of validity and stated assumptions. They do not reimplement the metrics' formulae and are not a substitute for the primary literature; parameter-level definitions should be confirmed with the metric authors before operational use. The ORDEM-derived indicator is constructed for demonstration and is not a published index. The orthogonality assertion between the two partitions is evidenced by the cited ESA and NASA study rather than asserted by us. This is a demonstrator built to show that the composition problem is mechanically checkable, and it is deliberately small.
Why we could build it quickly
Because the method was not invented here. This is the third domain in which we have applied the same pattern: formalise each authoritative representation's commitments machine-readably, compute where a mapping is sound, lossy or invalid, and emit a certificate a reviewer can check independently rather than having to trust us. Previously for defence and security information exchange, in the IES to HQDM crosswalk with its reasoner-certified 21-axiom bridge, and for UK government digital twin semantics through the National Digital Twin Programme.
The domains differ. The problem does not: two or more authoritative representations of the same reality, built on different assumptions, that somebody now needs to use together.
Reproduce it
The repository is public and MIT licensed. One of the tests asserts that the factor 2.3 to 3.0 divergence appears in the refusal reason, so if a future change ever caused the tool to quietly average the two models instead of refusing them, that test fails.
git clone https://github.com/fabio-rovai/space-metrics-crosswalk
cd space-metrics-crosswalk
pip install -r requirements.txt
python examples/master_ordem_1cm.py
pytest tests/ -q # 9 passedView the repository on GitHubReferences
- ESA and NASA, Flux comparison of MASTER-8 and ORDEM 3.1 modelled space debris population, 8th European Conference on Space Debris, Darmstadt, 2021. NASA NTRS 20210011563
- Letizia, Colombo, Lewis and Krag, on the ECOB space debris index
- Rossi, Valsecchi and Alessi, The Criticality of Spacecraft Index
- Space Sustainability Rating (World Economic Forum, ESA, MIT Space Enabled, University of Texas at Austin, BryceTech)
- Rovetto, An Ontological Architecture for Orbital Debris Data; Rovetto and Kelso, Preliminaries of a Space Situational Awareness Ontology
- UN COPUOS, Guidelines for the Long-term Sustainability of Outer Space Activities
