# Identification and Honest Recovery from Semantic Observation Kernels: Operator Error, Coarsening, and Stability

arXiv: https://arxiv.org/abs/2607.23130 · HTML: https://mlsanity.com/paper/2607.23130/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 16 of its 42 references that are in the map.
- First posted on arXiv 25 Jul 2026.

## Builds on (most cited first, 16 of 16)

- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [Semantic Uncertainty: Linguistic Invariances for Uncertainty Estimation in Natural Language Generation](https://mlsanity.com/paper/2302.09664/) (2023)
- [Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedback](https://mlsanity.com/paper/2305.14975/) (2023)
- [Distribution-Free Predictive Inference For Regression](https://mlsanity.com/paper/1604.04173/) (2016)
- [How Can We Know When Language Models Know? On the Calibration of Language Models for Question Answering](https://mlsanity.com/paper/2012.00955/) (2020)
- [Conformal prediction beyond exchangeability](https://mlsanity.com/paper/2202.13415/) (2022)
- [Predictive inference with the jackknife+](https://mlsanity.com/paper/1905.02928/) (2019)
- [Evaluating model calibration in classification](https://mlsanity.com/paper/1902.06977/) (2019)
- [A Survey of Uncertainty Estimation Methods on Large Language Models](https://mlsanity.com/paper/2503.00172/) (2025)
- [Calibration, Entropy Rates, and Memory in Language Models](https://mlsanity.com/paper/1906.05664/) (2019)
- [Minimax Confidence Intervals for the Sliced Wasserstein Distance](https://mlsanity.com/paper/1909.07862/) (2019)
- [On Subjective Uncertainty Quantification and Calibration in Natural Language Generation](https://mlsanity.com/paper/2406.05213/) (2024)
- [Honest calibration assessment for binary outcome predictions](https://mlsanity.com/paper/2203.04065/) (2022)
- [Calibrating Semantic Uncertainty from Observable Language-Model Probabilities](https://mlsanity.com/paper/2607.17447/) (2026)
- [Regularizing Double Machine Learning in Partially Linear Endogenous Models](https://mlsanity.com/paper/2101.12525/) (2021)
- [Calibrated Large Language Models for Binary Question Answering](https://mlsanity.com/paper/2407.01122/) (2024)

## Abstract (arXiv)

Probabilistic text generators, such as large language models, assign probabilities to phrases, but consequential decisions require posterior uncertainty over meaningful states. These are not interchangeable: language probabilities depend on the prompt, may be incomplete and need not reliably identify state uncertainty. Without a statistical bridge, fluent responses and numerical confidence are insufficient for inference or governance. We formulate recovery of the target posterior as a semiparametric inverse problem and develop honest recovery guarantees that account jointly for calibration error, measurement noise, incomplete probabilities and weak identification. Simulations demonstrate the predicted coverage and stability behaviour, while two frozen language-model studies demonstrate held-out recovery. The resulting method determines when a semantic measurement can be trusted for inference and when use, review, recalibration or abstention is warranted, providing a statistical foundation for runtime AI governance.