Data to 14 Oct 2026

Calibrating Deep Neural Networks using Focal Loss

arXiv 2002.09437 ↗First posted 21 Feb 2020cs.LG cs.CV stat.MLcited by 357

Not on the map. Why?

Abstract (arXiv)

Miscalibration - a mismatch between a model's confidence and its correctness - of Deep Neural Networks (DNNs) makes their predictions hard to rely on. Ideally, we want networks to be accurate, calibrated and confident. We show that, as opposed to the standard cross-entropy loss, focal loss [Lin et. al., 2017] allows us to learn models that are already very well calibrated. When combined with temperature scaling, whilst preserving accuracy, it yields state-of-the-art calibrated models. We provide a thorough analysis of the factors causing miscalibration, and use the insights we glean from this to justify the empirically excellent performance of focal loss. To facilitate the use of focal loss in practice, we also provide a principled approach to automatically select the hyperparameter involved in the loss function. We perform extensive experiments on a variety of computer vision and NLP datasets, and with a wide variety of network architectures, and show that our approach achieves state-of-the-art calibration without compromising on accuracy in almost all cases. Code is available at https://github.com/torrvision/focal_calibration.

Key facts. Cited by 357 papers in the map (data through 14 Oct 2026). First posted on arXiv 21 Feb 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 32; 2022: 65; 2023: 80; 2024: 74; 2025: 62; 2026: 44.

357 papers.

Sort:

By area:

2022Teaching Models to Express Their Uncertainty in Wordscited by 751
2023A Survey of Confidence Estimation and Calibration in Large Language Modelscited by 256
2022Mitigating Neural Network Overconfidence with Logit Normalizationcited by 252
2021Improving Calibration for Long-Tailed Recognitioncited by 212
2021On Calibration and Out-of-domain Generalizationcited by 134
2024Taming Overconfidence in LLMs: Reward Calibration in RLHFcited by 112
2022Uncertainty Quantification with Pre-trained Language Models: A Large-Scale Empirical Analysiscited by 104
2021Evidential Deep Learning for Open Set Action Recognitioncited by 96
2021Soft Calibration Objectives for Neural Networkscited by 92
2021The Devil is in the Margin: Margin-based Label Smoothing for Network Calibrationcited by 91
2022Reliable Visual Question Answering: Abstain Rather Than Answer Incorrectlycited by 89
2024C-TPT: Calibrated Test-Time Prompt Tuning for Vision-Language Models via Text Feature Dispersioncited by 83

38 more

2022Are All Losses Created Equal: A Neural Collapse Perspectivecited by 78
2023Calibration in Deep Learning: A Survey of the State-of-the-Artcited by 75
2022Training Uncertainty-Aware Classifiers with Conformalized Deep Learningcited by 64
2022Test-Time Adaptation via Self-Training with Nearest Neighbor Informationcited by 64
2023Not All Neuro-Symbolic Concepts Are Created Equal: Analysis and Mitigation of Reasoning Shortcutscited by 63
2022A Stitch in Time Saves Nine: A Train-Time Regularizing Loss for Improved Neural Network Calibrationcited by 49
2022Sample-dependent Adaptive Temperature Scaling for Improved Calibrationcited by 48
2022A Consistent and Differentiable Lp Canonical Calibration Error Estimatorcited by 47
2021Top-label calibration and multiclass-to-binary reductionscited by 46
2021BERT & Family Eat Word Salad: Experiments with Text Understandingcited by 44
2023Uncertainty in Natural Language Processing: Sources, Quantification, and Applicationscited by 44
2022Neural Collapse Inspired Attraction-Repulsion-Balanced Loss for Imbalanced Learningcited by 43
2023Rethinking Confidence Calibration for Failure Predictioncited by 43
2021Don't Just Blame Over-parametrization for Over-confidence: Theoretical Analysis of Calibration in Binary Classificationcited by 39
2021Trustworthy Multimodal Regression with Mixture of Normal-inverse Gamma Distributionscited by 39
2023OpenMix: Exploring Outlier Samples for Misclassification Detectioncited by 38
2024Thermometer: Towards Universal Calibration for Large Language Modelscited by 38
2024BEARS Make Neuro-Symbolic Models Aware of their Reasoning Shortcutscited by 37
2023When Does Optimizing a Proper Loss Yield Calibration?cited by 36
2023Towards Calibrated Robust Fine-Tuning of Vision-Language Modelscited by 35
2022Calibrated ensembles can mitigate accuracy tradeoffs under distribution shiftcited by 34
2022An Impartial Take to the CNN vs Transformer Robustness Contestcited by 34
2022Calibrated Selective Classificationcited by 33
2023Continuous Learning for Android Malware Detectioncited by 33
2023Dual Focal Loss for Calibrationcited by 33
2022AdaFocal: Calibration-aware Adaptive Focal Losscited by 32
2022Block Selection Method for Using Feature Norm in Out-of-distribution Detectioncited by 30
2021Meta-Calibration: Learning of Model Calibration Using Differentiable Expected Calibration Errorcited by 29
2024Robust Calibration of Large Vision-Language Adapterscited by 28
2022RegMixup: Mixup as a Regularizer Can Surprisingly Improve Accuracy and Out Distribution Robustnesscited by 27
2022Towards Improving Calibration in Object Detection Under Domain Shiftcited by 26
2024MapDistill: Boosting Efficient Camera-based HD Map Construction via Camera-LiDAR Fusion Model Distillationcited by 26
2021Diversifying Dialog Generation via Adaptive Label Smoothingcited by 25
2022Exploring Predictive Uncertainty and Calibration in NLP: A Study on the Impact of Method & Data Scarcitycited by 25
2022On Calibrating Semantic Segmentation Models: Analyses and An Algorithmcited by 25
2023Beyond Confidence: Reliable Models Should Also Consider Atypicalitycited by 25
2025Restoring Calibration for Aligned Large Language Models: A Calibration-Aware Fine-Tuning Approachcited by 25
2022MUAD: Multiple Uncertainties for Autonomous Driving, a benchmark for multiple uncertainty types and taskscited by 24

50 most cited of 357 All 357 (JSON)