Foundational Research: Trustworthy Scientific AI
Marginal coverage is not joint coverage
A scientific model rarely predicts one number. It predicts a vector, a material's formation energy, band gap and stability, a protein's per-residue confidences, and those outputs are correlated. Calibrate the uncertainty one output at a time, as conformal prediction usually does, and you get a per-target guarantee that looks right and a joint guarantee that is quietly wrong. Measured on real materials data, a "90%" independent certificate delivers 79% joint coverage. Here is why, and what fixes it.
Independent certificate
90% → 79%
claimed vs actual joint coverage
Coupled certificate
89.7%
joint coverage restored
Evidence
1,600
real OQMD DFT materials
The Challenge
Foundation and scientific models are deployed with uncertainty that is either uncalibrated or calibrated marginally, one output at a time. But a deployment decision usually depends on the whole prediction being right at once: is this material stable and low-formation-energy and a semiconductor? A certificate that has to hold jointly, the realized outcome vector falling inside an emitted region at the claimed rate, cannot be assembled from independent per-target intervals when the outputs are correlated. Stack d marginal 90% intervals and the joint region is the wrong shape and the wrong size.
The measurement
We took 1,600 real materials from the Open Quantum Materials Database, three DFT targets, and a gradient-boosted regressor trained once and frozen. Every certificate below is calibrated to a nominal 90% joint coverage on a held-out split, and measured on a test split. The residuals are genuinely correlated (formation energy and stability +0.50), which is exactly why the joint problem is not the marginal problem.
| Method | Marginal coverage | Joint coverage | Region size |
|---|---|---|---|
| Independent conformal (90%/target) | 0.902 | 0.792 | 1.0x |
| Bonferroni conformal | 0.960 | 0.907 | 6.18x |
| Coupled conformal (max-score box) | 0.954 | 0.897 | 4.83x |
| Gaussian-copula region | joint by design | 0.892 | 0.43x of coupled |
Independent conformal is marginally correct (90.2% per target) and jointly wrong (79.2%). Bonferroni restores joint validity but at 6.18x the volume, intervals too wide to act on. The coupled certificate hits 89.7% joint coverage at 22% less volume than Bonferroni, and the Gaussian copula, using the dependence rather than ignoring it, matches the joint coverage in a region 57% smaller than the coupled box.
Why this is the ownable gap
The fix is not a bigger model or more calibration data; independent conformal already has all the coverage it was asked for, marginally. The fix is to calibrate the object you actually deploy: the joint region. A global max-score conformal certificate does that with a finite-sample guarantee and no distributional assumption; a copula does it with a model of the dependence, buying back most of the volume Bonferroni throws away. Both emit a machine-checkable region whose soundness is the measured joint coverage, reported here on held-out data. This is the uncertainty companion to our WorldKernel coupling work, and it ladders to vine copulas over the outputs of several frozen models, which is where the copula-net world-model program goes next.
"A model that reports 90% confidence on each of three correlated outputs is not 90% confident in the three together. Deploy on the marginal number and you are wrong about the joint one in the exact cases that matter. Calibrate the region, not the axes."
Fabio Rovai, Tesseract Academy
An independent, self-initiated study on open data and open tools: OQMD's DFT properties under their open licence. The frozen model is a gradient-boosted regressor standing in for a large scientific model; the conformal guarantees are distribution-free, so the coverage result transfers, and running the wrapper on ESM-2 or a Materials Project foundation model is the honest scale-up. The copula is Gaussian, a first tractable dependence model; the region-size comparison is stated precisely in the build report. This is a coverage phenomenon on real data, not a new theorem.
Reproduce it
Fetch real OQMD data, train and freeze the model, run every certificate, in Colab or the repo.
