# Confidence Calibration for Intent Detection via Hyperspherical Space and Rebalanced Accuracy-Uncertainty Loss

arXiv: https://arxiv.org/abs/2203.09278 · HTML: https://mlsanity.com/paper/2203.09278/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 36 of its 53 references that are in the map.
- First posted on arXiv 17 Mar 2022.

## Builds on (most cited first, 30 of 36)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [XLNet: Generalized Autoregressive Pretraining for Language Understanding](https://mlsanity.com/paper/1906.08237/) (2019)
- [ALBERT: A Lite BERT for Self-supervised Learning of Language Representations](https://mlsanity.com/paper/1909.11942/) (2019)
- [When Does Label Smoothing Help?](https://mlsanity.com/paper/1906.02629/) (2019)
- [Deep Neural Networks are Easily Fooled: High Confidence Predictions for Unrecognizable Images](https://mlsanity.com/paper/1412.1897/) (2014)
- [Regularizing Neural Networks by Penalizing Confident Output Distributions](https://mlsanity.com/paper/1701.06548/) (2017)
- [Efficient Intent Detection with Dual Sentence Encoders](https://mlsanity.com/paper/2003.04807/) (2020)
- [An Evaluation Dataset for Intent Classification and Out-of-Scope Prediction](https://mlsanity.com/paper/1909.02027/) (2019)
- [Beyond temperature scaling: Obtaining well-calibrated multiclass probabilities with Dirichlet calibration](https://mlsanity.com/paper/1910.12656/) (2019)
- [Calibrating Deep Neural Networks using Focal Loss](https://mlsanity.com/paper/2002.09437/) (2020)
- [Verified Uncertainty Calibration](https://mlsanity.com/paper/1909.10155/) (2019)
- [CLUE: A Chinese Language Understanding Evaluation Benchmark](https://mlsanity.com/paper/2004.05986/) (2020)
- [Mix-n-Match: Ensemble and Compositional Methods for Uncertainty Calibration in Deep Learning](https://mlsanity.com/paper/2003.07329/) (2020)
- [Benchmarking Natural Language Understanding Services for building Conversational Agents](https://mlsanity.com/paper/1903.05566/) (2019)
- [Attention-Based Recurrent Neural Network Models for Joint Intent Detection and Slot Filling](https://mlsanity.com/paper/1609.01454/) (2016)
- [Improving model calibration with accuracy versus uncertainty optimization](https://mlsanity.com/paper/2012.07923/) (2020)
- [Multivariate Confidence Calibration for Object Detection](https://mlsanity.com/paper/2004.13546/) (2020)
- [Calibration of Neural Networks using Splines](https://mlsanity.com/paper/2006.12800/) (2020)
- [A Novel Bi-directional Interrelated Model for Joint Intent Detection and Slot Filling](https://mlsanity.com/paper/1907.00390/) (2019)
- [Posterior calibration and exploratory analysis for natural language processing models](https://mlsanity.com/paper/1508.05154/) (2015)
- [Joint Slot Filling and Intent Detection via Capsule Neural Networks](https://mlsanity.com/paper/1812.09471/) (2018)
- [Intra Order-preserving Functions for Calibration of Multi-Class Neural Networks](https://mlsanity.com/paper/2003.06820/) (2020)
- [On the Inference Calibration of Neural Machine Translation](https://mlsanity.com/paper/2005.00963/) (2020)
- [Multilingual and Cross-Lingual Intent Detection from Spoken Data](https://mlsanity.com/paper/2104.08524/) (2021)
- [Regularization via Structural Label Smoothing](https://mlsanity.com/paper/2001.01900/) (2020)
- [Augmented Natural Language for Generative Sequence Labeling](https://mlsanity.com/paper/2009.13272/) (2020)
- [Example-Driven Intent Prediction with Observers](https://mlsanity.com/paper/2010.08684/) (2020)

## Built on by (most cited first, 2 of 2)

- [Intent-calibrated Self-training for Answer Selection in Open-domain Dialogues](https://mlsanity.com/paper/2307.06703/) (2023)
- [Uncertainty Estimation via Hyperspherical Confidence Mapping](https://mlsanity.com/paper/2605.05964/) (2026)

## Abstract (arXiv)

Data-driven methods have achieved notable performance on intent detection, which is a task to comprehend user queries. Nonetheless, they are controversial for over-confident predictions. In some scenarios, users do not only care about the accuracy but also the confidence of model. Unfortunately, mainstream neural networks are poorly calibrated, with a large gap between accuracy and confidence. To handle this problem defined as confidence calibration, we propose a model using the hyperspherical space and rebalanced accuracy-uncertainty loss. Specifically, we project the label vector onto hyperspherical space uniformly to generate a dense label representation matrix, which mitigates over-confident predictions due to overfitting sparce one-hot label matrix. Besides, we rebalance samples of different accuracy and uncertainty to better guide model training. Experiments on the open datasets verify that our model outperforms the existing calibration methods and achieves a significant improvement on the calibration metric.