mlsanityData to 13 Oct 2026

On Calibration of Modern Neural Networks

arXiv 1706.04599 ↗First posted 14 Jun 2017cs.LGcited by 4,902

Not on the map. Why?

Cited by 160 in Multimodal LLMs / vision-language models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Confidence calibration -- the problem of predicting probability estimates representative of the true correctness likelihood -- is important for classification models in many applications. We discover that modern neural networks, unlike those from a decade ago, are poorly calibrated. Through extensive experiments, we observe that depth, width, weight decay, and Batch Normalization are important factors influencing calibration. We evaluate the performance of various post-processing calibration methods on state-of-the-art architectures with image and document classification datasets. Our analysis and experiments not only offer insights into neural network learning, but also provide a simple and straightforward recipe for practical settings: on most datasets, temperature scaling -- a single-parameter variant of Platt Scaling -- is surprisingly effective at calibrating predictions.

Key facts. Cited by 4,902 papers in the map (data through 13 Oct 2026). First posted on arXiv 14 Jun 2017.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2018: 1; 2019: 0; 2020: 0; 2021: 534; 2022: 560; 2023: 700; 2024: 745; 2025: 823; 2026: 1539.

4,902 papers.

Sort:

By area:

  1. 2022Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedbackcited by 3,691
  2. 2022Language Models (Mostly) Know What They Knowcited by 1,823
  3. 2022Model soups: averaging weights of multiple fine-tuned models improves accuracy without increasing inference timecited by 1,384
  4. 2021Calibrate Before Use: Improving Few-Shot Performance of Language Modelscited by 1,376
  5. 2023A Survey on Evaluation of Large Language Modelscited by 1,236
  6. 2023Can LLMs Express Their Uncertainty? An Empirical Evaluation of Confidence Elicitation in LLMscited by 960
  7. 2023Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedbackcited by 800
  8. 2021Robust fine-tuning of zero-shot modelscited by 790
  9. 2023Whose Opinions Do Language Models Reflect?cited by 768
  10. 2023CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquingcited by 764
  11. 2022Teaching Models to Express Their Uncertainty in Wordscited by 740
  12. 2023Siren's Song in the AI Ocean: A Survey on Hallucination in Large Language Modelscited by 697
38 more
  1. 2023Scaling Vision Transformers to 22 Billion Parameterscited by 693
  2. 2021A Survey of Uncertainty in Deep Neural Networkscited by 676
  3. 2022Large Language Models Can Self-Improvecited by 659
  4. 2023GPTFUZZER: Red Teaming Large Language Models with Auto-Generated Jailbreak Promptscited by 556
  5. 2022Continual Test-Time Domain Adaptationcited by 523
  6. 2023End-to-end Autonomous Driving: Challenges and Frontierscited by 487
  7. 2018Scalable agent alignment via reward modeling: a research directioncited by 443
  8. 2021Dynamic Neural Networks: A Surveycited by 406
  9. 2021Revisiting the Calibration of Modern Neural Networkscited by 388
  10. 2021MEMO: Test Time Robustness via Adaptation and Augmentationcited by 376
  11. 2021Laplace Redux -- Effortless Bayesian Deep Learningcited by 361
  12. 2023A Comprehensive Survey on Test-Time Adaptation under Distribution Shiftscited by 355
  13. 2021Open-Set Recognition: a Good Closed-Set Classifier is All You Need?cited by 353
  14. 2021Skillful Precipitation Nowcasting using Deep Generative Models of Radarcited by 310
  15. 2021What Are Bayesian Neural Network Posteriors Really Like?cited by 307
  16. 2023Robots That Ask For Help: Uncertainty Alignment for Large Language Model Plannerscited by 302
  17. 2023ReConcile: Round-Table Conference Improves Reasoning via Consensus among Diverse LLMscited by 302
  18. 2021In Defense of Pseudo-Labeling: An Uncertainty-Aware Pseudo-label Selection Framework for Semi-Supervised Learningcited by 301
  19. 2021Unsolved Problems in ML Safetycited by 291
  20. 2021DyTox: Transformers for Continual Learning with DYnamic TOken eXpansioncited by 291
  21. 2021Exploring the Limits of Out-of-Distribution Detectioncited by 283
  22. 2023RAGTruth: A Hallucination Corpus for Developing Trustworthy Retrieval-Augmented Language Modelscited by 278
  23. 2022OpenOOD: Benchmarking Generalized Out-of-Distribution Detectioncited by 268
  24. 2022Contrastive Test-Time Adaptationcited by 259
  25. 2022Prompting GPT-3 To Be Reliablecited by 259
  26. 2023A Survey of Confidence Estimation and Calibration in Large Language Modelscited by 255
  27. 2022Mitigating Neural Network Overconfidence with Logit Normalizationcited by 251
  28. 2022FreeMatch: Self-adaptive Thresholding for Semi-supervised Learningcited by 232
  29. 2023Text2Motion: From Natural Language Instructions to Feasible Planscited by 223
  30. 2021Improving Calibration for Long-Tailed Recognitioncited by 212
  31. 2022Extremely Simple Activation Shaping for Out-of-Distribution Detectioncited by 212
  32. 2021Deep Deterministic Uncertainty: A Simple Baselinecited by 200
  33. 2023Self-Evaluation Guided Beam Search for Reasoningcited by 196
  34. 2022Metrics reloaded: Recommendations for image analysis validationcited by 192
  35. 2021Does Knowledge Distillation Really Work?cited by 188
  36. 2023OpenOOD v1.5: Enhanced Benchmark for Out-of-Distribution Detectioncited by 179
  37. 2024Don't Hallucinate, Abstain: Identifying LLM Knowledge Gaps via Multi-LLM Collaborationcited by 179
  38. 2022PLA: Language-Driven Open-Vocabulary 3D Scene Understandingcited by 176

50 most cited of 4,902 All 4,902 (JSON)