Skip to main content
Back to Use Cases

Case Study: Space Environment Measurement

Space debris metrics that cannot be added up

The world's two premier orbital debris models disagree by a factor of 2.3 to 3.0 at the diameter where collision-avoidance decisions are made. Neither is wrong. They divide the same physical population along axes that do not align, and nothing in either model's output records that fact. We built an open checker that catches it.

Divergence at 1 cm

2.3 to 3.0x

between two authoritative models

Standards used

3

QUDT, SOSA/SSN, PROV-O

Open and tested

9 / 9

tests passing, MIT licensed

The Challenge

ESA's MASTER-8 and NASA's ORDEM 3.1 are the two premier orbital debris engineering models. Both are authoritative. Both are carefully built. Both describe the same physical reality. On identical simulation cases they diverge sharply, and the joint ESA and NASA comparison study is explicit that this is not a matter of one being right: "each model provides its agency's best estimate of the orbital debris environment. One model may be more accurate in one regime while the other may be more accurate in another, and it is expected that the truth lies somewhere in between."

The structural cause is stated in the same paper, and it is the sentence that matters: "a direct comparison of the source models is not immediately possible from the model output." MASTER-8 divides the population by generative source, into fragments, solid rocket motor slag and dust, NaK droplets, paint flakes, ejecta and MLI fragments. ORDEM 3.1 divides it by material density, into low, medium, high, NaK and intacts. One population, orthogonal cuts, only the NaK component in common.

1 m

Near identical

Down to ~4 mm

Less than an order of magnitude, ORDEM 3.1 lower

1 cm

MASTER-8 higher by a factor of 2.3, and 3.0 in the second case

2 mm to 10 µm

ORDEM 3.1 higher by a factor of 2 to 10

Sub-millimetre

ORDEM 3.1 almost two orders of magnitude higher

The 1 cm row is the one that matters operationally. The comparison paper calls it "a critical diameter threshold for collision risk estimations", and it is where collision-avoidance thresholds, shielding specification and post-mission disposal requirements are set.

Why this is a safety problem, not a housekeeping problem

Every environmental indicator is computed over an underlying population model, and therefore silently inherits that model's partition. ECOB, the Criticality of Spacecraft Index and the Space Sustainability Rating mission index each carry an inheritance their outputs do not record. Combine two indicators built on different models and you have combined two incompatible partitions with no trace that it happened.

There is no error message. The composed indicator returns a number, and the number gets used. It is the same class of failure as a unit mismatch, and it has the same property: cheap to prevent by construction, effectively undetectable afterwards.

What we built

Each metric is declared as an explicit, machine-readable object rather than described in prose. We deliberately built on mature standardised semantic infrastructure rather than inventing anything: QUDT for quantity kinds and units, SOSA/SSN (a W3C Recommendation) for observation semantics, and PROV-O (a W3C Recommendation) for the assumption and derivation chain. The vocabulary adds only the two things those three do not supply: the population partition a metric inherits from its source model, and its regime of validity.

R1Dimension

Quantity kinds must match for additive composition. Carried by QUDT.

R2Denominator

Quantities normalised against different bases are not additive. Per mission and per resident space object are not interchangeable.

R3Partition

Source models that partition the same population along orthogonal axes cannot have their outputs arithmetically combined. This is the rule the domain currently lacks.

R4Regime of validity

Declared size ranges and orbital regimes must overlap, and any requested evaluation point must sit inside the intersection.

R5Independence

Metrics sharing a source model are flagged as non-independent: common-mode error means a composed figure understates uncertainty.

The decision is symbolic. There is no statistical component and no language model anywhere in the decision path, so the same inputs always produce the same, inspectable answer.

Four verdicts, not one refusal

CompositionVerdictWhy
ECOB + ORDEM-derived indicatorINVALIDR3, orthogonal partitions
ECOB + Criticality of Spacecraft IndexINVALIDR2, per mission against per object
ECOB + SSR Mission IndexVALID WITH CAVEATR5, both derive from MASTER-8, so not independent
Any pair, reported side by sideVALIDNo arithmetic performed

The third row is the one that shows the tool is doing real work. A checker that refuses everything is useless, and one that permits everything is dangerous. ECOB and the SSR mission index genuinely are structurally combinable, and the framework permits the combination while flagging that both derive from MASTER-8 and are therefore not independent estimates. Shared source-population error is common-mode, so a composed figure will understate its own uncertainty. That is a judgement a spreadsheet cannot make and a dashboard will not surface.

Every refusal is issued together with what may be done instead: report both figures side by side attributed to their source models with the divergence stated; use either metric alone within its own declared regime of validity; or establish an explicit, evidenced mapping between the two partitions, after which composition becomes checkable. The purpose is to establish what can validly be done. Refusal is the safety property underneath, not the headline feature.

The gap this sits in

A decade of orbital debris ontology work exists in the literature. The Space Object Ontology, the Space Situational Awareness Ontology and an associated orbital event ontology are all formalised in OWL, and all are described by their authors as under development. None has been mapped to an operational data standard or to an engineering model.

Meanwhile the CCSDS orbit, conjunction and tracking message families are mature and operationally adopted, but they are syntactic: they standardise message structure, not the meaning of a derived quantity, not the population partition it was computed over, and not the assumptions under which it holds. Four layers, developed independently, none mapped to the next. This artifact is one small, concrete step across that gap.

Scope, stated plainly

The declarations capture the structural commitments of each metric: quantity kind, denominator, source model, inherited partition, regime of validity and stated assumptions. They do not reimplement the metrics' formulae and are not a substitute for the primary literature; parameter-level definitions should be confirmed with the metric authors before operational use. The ORDEM-derived indicator is constructed for demonstration and is not a published index. The orthogonality assertion between the two partitions is evidenced by the cited ESA and NASA study rather than asserted by us. This is a demonstrator built to show that the composition problem is mechanically checkable, and it is deliberately small.

Why we could build it quickly

Because the method was not invented here. This is the third domain in which we have applied the same pattern: formalise each authoritative representation's commitments machine-readably, compute where a mapping is sound, lossy or invalid, and emit a certificate a reviewer can check independently rather than having to trust us. Previously for defence and security information exchange, in the IES to HQDM crosswalk with its reasoner-certified 21-axiom bridge, and for UK government digital twin semantics through the National Digital Twin Programme.

The domains differ. The problem does not: two or more authoritative representations of the same reality, built on different assumptions, that somebody now needs to use together.

Reproduce it

The repository is public and MIT licensed. One of the tests asserts that the factor 2.3 to 3.0 divergence appears in the refusal reason, so if a future change ever caused the tool to quietly average the two models instead of refusing them, that test fails.

git clone https://github.com/fabio-rovai/space-metrics-crosswalk
cd space-metrics-crosswalk
pip install -r requirements.txt
python examples/master_ordem_1cm.py
pytest tests/ -q          # 9 passed
View the repository on GitHub

References

  • ESA and NASA, Flux comparison of MASTER-8 and ORDEM 3.1 modelled space debris population, 8th European Conference on Space Debris, Darmstadt, 2021. NASA NTRS 20210011563
  • Letizia, Colombo, Lewis and Krag, on the ECOB space debris index
  • Rossi, Valsecchi and Alessi, The Criticality of Spacecraft Index
  • Space Sustainability Rating (World Economic Forum, ESA, MIT Space Enabled, University of Texas at Austin, BryceTech)
  • Rovetto, An Ontological Architecture for Orbital Debris Data; Rovetto and Kelso, Preliminaries of a Space Situational Awareness Ontology
  • UN COPUOS, Guidelines for the Long-term Sustainability of Outer Space Activities