mlsanityData to 14 Oct 2026

Posterior calibration and exploratory analysis for natural language processing models

arXiv 1508.05154 ↗First posted 21 Aug 2015cs.CLcited by 76

Not on the map. Why?

Abstract (arXiv)

Many models in natural language processing define probabilistic distributions over linguistic structures. We argue that (1) the quality of a model' s posterior distribution can and should be directly evaluated, as to whether probabilities correspond to empirical frequencies, and (2) NLP uncertainty can be projected not only to pipeline components, but also to exploratory data analysis, telling a user when to trust and not trust the NLP analysis. We present a method to analyze calibration, and apply it to compare the miscalibration of several commonly used models. We also contribute a coreference sampling algorithm that can create confidence intervals for a political event extraction task.

Key facts. Cited by 76 papers in the map (data through 14 Oct 2026). First posted on arXiv 21 Aug 2015.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 11; 2022: 20; 2023: 17; 2024: 12; 2025: 13; 2026: 3.

76 papers.

Sort:

By area:

  1. 2022Language Models (Mostly) Know What They Knowcited by 1,856
  2. 2023CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquingcited by 783
  3. 2022Teaching Models to Express Their Uncertainty in Wordscited by 751
  4. 2021Unsolved Problems in ML Safetycited by 292
  5. 2021PixMix: Dreamlike Pictures Comprehensively Improve Safety Measurescited by 131
  6. 2023On the Calibration of Large Language Models and Alignmentcited by 73
  7. 2022On the Relation between Sensitivity and Accuracy in In-context Learningcited by 72
  8. 2024API Is Enough: Conformal Prediction for Large Language Models Without Logit-Accesscited by 69
  9. 2022Better Uncertainty Calibration via Proper Scores for Classification and Beyondcited by 65
  10. 2022Forecasting Future World Events with Neural Networkscited by 63
  11. 2021Knowing More About Questions Can Help: Improving Calibration in Question Answeringcited by 47
  12. 2022A Consistent and Differentiable Lp Canonical Calibration Error Estimatorcited by 47
38 more
  1. 2021Deep Cox Mixtures for Survival Regressioncited by 44
  2. 2023LitCab: Lightweight Language Model Calibration over Short- and Long-form Responsescited by 44
  3. 2024A Closer Look at the Robustness of Contrastive Language-Image Pre-Training (CLIP)cited by 44
  4. 2021Modeling Fine-Grained Entity Types with Box Embeddingscited by 36
  5. 2021Calibrated and Sharp Uncertainties in Deep Learning via Density Estimationcited by 33
  6. 2023Dual Focal Loss for Calibrationcited by 33
  7. 2022AdaFocal: Calibration-aware Adaptive Focal Losscited by 32
  8. 2021Distributed NLI: Learning to Predict Human Opinion Distributions for Language Reasoningcited by 31
  9. 2022On the Calibration of Pre-trained Language Models using Mixup Guided by Area Under the Margin and Saliencycited by 31
  10. 2024On Calibration of LLM-based Guard Models for Reliable Content Moderationcited by 23
  11. 2022You Only Cut Once: Boosting Data Augmentation with a Single Cutcited by 22
  12. 2022Interpretable Self-Aware Neural Networks for Robust Trajectory Predictioncited by 22
  13. 2022A Stable, Fast, and Fully Automatic Learning Algorithm for Predictive Coding Networkscited by 22
  14. 2024An Empirical Study Into What Matters for Calibrating Vision-Language Modelscited by 20
  15. 2022Concept-based Explanations for Out-Of-Distribution Detectorscited by 16
  16. 2023Reliability in Semantic Segmentation: Are We on the Right Track?cited by 16
  17. 2024Probabilities of Chat LLMs Are Miscalibrated but Still Predict Correctness on Multiple-Choice Q&Acited by 16
  18. 2024Dynamic Correlation Learning and Regularization for Multi-Label Confidence Calibrationcited by 13
  19. 2024Toward a Holistic Evaluation of Robustness in CLIP Modelscited by 13
  20. 2023Where's the Liability in Harmful AI Speech?cited by 12
  21. 2021Should Ensemble Members Be Calibrated?cited by 11
  22. 2023Consistent and Asymptotically Unbiased Estimation of Proper Calibration Errorscited by 11
  23. 2021Online Calibrated and Conformal Prediction Improves Bayesian Optimizationcited by 10
  24. 2023Minimum-Risk Recalibration of Classifierscited by 10
  25. 2022CropMix: Sampling a Rich Input Distribution via Multi-Scale Croppingcited by 8
  26. 2025Learning to Insert [PAUSE] Tokens for Better Reasoningcited by 8
  27. 2024Calibrating Expressions of Certaintycited by 7
  28. 2021Capturing Label Distribution: A Case Study in NLIcited by 6
  29. 2022Calibration of Natural Language Understanding Models with Venn--ABERS Predictorscited by 6
  30. 2023MaxEnt Loss: Constrained Maximum Entropy for Calibration under Out-of-Distribution Shiftcited by 6
  31. 2022What Images are More Memorable to Machines?cited by 5
  32. 2023Understanding Calibration of Deep Neural Networks for Medical Image Classificationcited by 5
  33. 2024Towards Certification of Uncertainty Calibration under Adversarial Attackscited by 5
  34. 2025GRACE: A Granular Benchmark for Evaluating Model Calibration against Human Calibrationcited by 5
  35. 2025Uncertainty Weighted Gradients for Model Calibrationcited by 5
  36. 2025Comparing Uncertainty Measurement and Mitigation Methods for Large Language Models: A Systematic Reviewcited by 4
  37. 2023Model Calibration in Dense Classification with Adaptive Label Perturbationcited by 3
  38. 2023Understanding Data Augmentation from a Robustness Perspectivecited by 3

50 most cited of 76 All 76 (JSON)