# Efficient Uncertainty Estimation with Gaussian Process for Reliable Dialog Response Retrieval

arXiv: https://arxiv.org/abs/2303.08599 · HTML: https://mlsanity.com/paper/2303.08599/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 21 references that are in the map.
- First posted on arXiv 15 Mar 2023.

## Builds on (most cited first, 18 of 18)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning](https://mlsanity.com/paper/1506.02142/) (2015)
- [Spectral Normalization for Generative Adversarial Networks](https://mlsanity.com/paper/1802.05957/) (2018)
- [Deep Neural Networks as Gaussian Processes](https://mlsanity.com/paper/1711.00165/) (2017)
- [Simple and Principled Uncertainty Estimation with Deterministic Deep Learning via Distance Awareness](https://mlsanity.com/paper/2006.10108/) (2020)
- [Calibrating Deep Neural Networks using Focal Loss](https://mlsanity.com/paper/2002.09437/) (2020)
- [Selective Question Answering under Domain Shift](https://mlsanity.com/paper/2006.09462/) (2020)
- [Being Bayesian, Even Just a Bit, Fixes Overconfidence in ReLU Networks](https://mlsanity.com/paper/2002.10118/) (2020)
- [The Ubuntu Dialogue Corpus: A Large Dataset for Research in Unstructured Multi-Turn Dialogue Systems](https://mlsanity.com/paper/1506.08909/) (2015)
- [Masksembles for Uncertainty Estimation](https://mlsanity.com/paper/2012.08334/) (2020)
- [On Feature Collapse and Deep Kernel Learning for Single Forward Pass Uncertainty](https://mlsanity.com/paper/2102.11409/) (2021)
- [Analyzing and Characterizing User Intent in Information-seeking Conversations](https://mlsanity.com/paper/1804.08759/) (2018)
- [Introducing MANtIS: a novel Multi-Domain Information Seeking Dialogues Dataset](https://mlsanity.com/paper/1912.04639/) (2019)
- [Deep Neural Networks as Point Estimates for Deep Gaussian Processes](https://mlsanity.com/paper/2105.04504/) (2021)
- [On the Calibration and Uncertainty of Neural Learning to Rank Models](https://mlsanity.com/paper/2101.04356/) (2021)
- [Not All Relevance Scores are Equal: Efficient Uncertainty and Calibration Modeling for Deep Retrieval Models](https://mlsanity.com/paper/2105.04651/) (2021)
- [Mean-Field Approximation to Gaussian-Softmax Integral with Application to Uncertainty Estimation](https://mlsanity.com/paper/2006.07584/) (2020)

## Built on by (most cited first, 2 of 2)

- [Trimming the Risk: Towards Reliable Continuous Training for Deep Learning Inspection Systems](https://mlsanity.com/paper/2409.09108/) (2024)
- [ALF: Advertiser Large Foundation Model for Multi-Modal Advertiser Understanding](https://mlsanity.com/paper/2504.18785/) (2025)

## Abstract (arXiv)

Deep neural networks have achieved remarkable performance in retrieval-based dialogue systems, but they are shown to be ill calibrated. Though basic calibration methods like Monte Carlo Dropout and Ensemble can calibrate well, these methods are time-consuming in the training or inference stages. To tackle these challenges, we propose an efficient uncertainty calibration framework GPF-BERT for BERT-based conversational search, which employs a Gaussian Process layer and the focal loss on top of the BERT architecture to achieve a high-quality neural ranker. Extensive experiments are conducted to verify the effectiveness of our method. In comparison with basic calibration methods, GPF-BERT achieves the lowest empirical calibration error (ECE) in three in-domain datasets and the distributional shift tasks, while yielding the highest R10@1 and MAP performance on most cases. In terms of time consumption, our GPF-BERT has an 8× speedup.