mlsanityData to 14 Oct 2026

Identification and Honest Recovery from Semantic Observation Kernels: Operator Error, Coarsening, and Stability

arXiv 2607.23130 ↗First posted 25 Jul 2026stat.ME stat.MLcited by 0

Not on the map. Why?

Abstract (arXiv)

Probabilistic text generators, such as large language models, assign probabilities to phrases, but consequential decisions require posterior uncertainty over meaningful states. These are not interchangeable: language probabilities depend on the prompt, may be incomplete and need not reliably identify state uncertainty. Without a statistical bridge, fluent responses and numerical confidence are insufficient for inference or governance. We formulate recovery of the target posterior as a semiparametric inverse problem and develop honest recovery guarantees that account jointly for calibration error, measurement noise, incomplete probabilities and weak identification. Simulations demonstrate the predicted coverage and stability behaviour, while two frozen language-model studies demonstrate held-out recovery. The resulting method determines when a semantic measurement can be trusted for inference and when use, review, recalibration or abstention is warranted, providing a statistical foundation for runtime AI governance.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 16 of its 42 references that are in the map. First posted on arXiv 25 Jul 2026.

Builds on

16 of 42 references.

References · 16

  1. 2017On Calibration of Modern Neural Networkscited by 4,960
  2. 2023Semantic Uncertainty: Linguistic Invariances for Uncertainty Estimation in Natural Language Generationcited by 819
  3. 2023Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedbackcited by 812
  4. 2016Distribution-Free Predictive Inference For Regressioncited by 668
  5. 2020How Can We Know When Language Models Know? On the Calibration of Language Models for Question Answeringcited by 445
  6. 2022Conformal prediction beyond exchangeabilitycited by 355
  7. 2019Predictive inference with the jackknife+cited by 211
  8. 2019Evaluating model calibration in classificationcited by 168
  9. 2025A Survey of Uncertainty Estimation Methods on Large Language Modelscited by 62
  10. 2019Calibration, Entropy Rates, and Memory in Language Modelscited by 32
  11. 2019Minimax Confidence Intervals for the Sliced Wasserstein Distancecited by 24
  12. 2024On Subjective Uncertainty Quantification and Calibration in Natural Language Generationcited by 16
4 more
  1. 2022Honest calibration assessment for binary outcome predictionscited by 8
  2. 2026Calibrating Semantic Uncertainty from Observable Language-Model Probabilitiescited by 4
  3. 2021Regularizing Double Machine Learning in Partially Linear Endogenous Modelscited by 3
  4. 2024Calibrated Large Language Models for Binary Question Answeringcited by 3

Built on by

Not cited yet.