# Confidence Scoring Using Whitebox Meta-models with Linear Classifier Probes

arXiv: https://arxiv.org/abs/1805.05396 · HTML: https://mlsanity.com/paper/1805.05396/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 May 2018.

## Built on by (most cited first, 21 of 21)

- [AI Transparency in the Age of LLMs: A Human-Centered Research Roadmap](https://mlsanity.com/paper/2306.01941/) (2023)
- [Post-hoc Uncertainty Learning using a Dirichlet Meta-Model](https://mlsanity.com/paper/2212.07359/) (2022)
- [Uncertainty Quantification 360: A Holistic Toolkit for Quantifying and Communicating the Uncertainty of AI](https://mlsanity.com/paper/2106.01410/) (2021)
- [An evaluation of word-level confidence estimation for end-to-end automatic speech recognition](https://mlsanity.com/paper/2101.05525/) (2021)
- [MetaToken: Detecting Hallucination in Image Descriptions by Meta Classification](https://mlsanity.com/paper/2405.19186/) (2024)
- [Confidence-based Estimators for Predictive Performance in Model Monitoring](https://mlsanity.com/paper/2407.08649/) (2024)
- [A Survey on Uncertainty Toolkits for Deep Learning](https://mlsanity.com/paper/2205.01040/) (2022)
- [Uncertainty Characteristics Curves: A Systematic Assessment of Prediction Intervals](https://mlsanity.com/paper/2106.00858/) (2021)
- [Performance Prediction Under Dataset Shift](https://mlsanity.com/paper/2206.10697/) (2022)
- [Out-of-Distribution Example Detection in Deep Neural Networks using Distance to Modelled Embedding](https://mlsanity.com/paper/2108.10673/) (2021)
- [Bi-directional Model Cascading with Proxy Confidence](https://mlsanity.com/paper/2504.19391/) (2025)
- [SurvUnc: A Meta-Model Based Uncertainty Quantification Framework for Survival Analysis](https://mlsanity.com/paper/2505.14803/) (2025)
- [Efficient Contrastive Decoding with Probabilistic Hallucination Detection - Mitigating Hallucinations in Large Vision Language Models -](https://mlsanity.com/paper/2504.12137/) (2025)
- [Neurosymbolic Artificial Intelligence for Robust Network Intrusion Detection: From Scratch to Transfer Learning](https://mlsanity.com/paper/2506.04454/) (2025)
- [Uncertainty Surrogates for Deep Learning](https://mlsanity.com/paper/2104.08147/) (2021)
- [Uncertainty Quantification for Rule-Based Models](https://mlsanity.com/paper/2211.01915/) (2022)
- [Assessment of Prediction Intervals Using Uncertainty Characteristics Curves](https://mlsanity.com/paper/2310.03158/) (2023)
- [Neurosymbolic AI Transfer Learning Improves Network Intrusion Detection](https://mlsanity.com/paper/2509.10850/) (2025)
- [Guided Uncertainty Learning Using a Post-Hoc Evidential Meta-Model](https://mlsanity.com/paper/2509.24492/) (2025)
- [UA-Net: Uncertainty-Aware Network for TRISO Image Semantic Segmentation](https://mlsanity.com/paper/2604.15542/) (2026)
- [MetaErr: Towards Predicting Error Patterns in Deep Neural Networks](https://mlsanity.com/paper/2604.23289/) (2026)

## Abstract (arXiv)

We propose a novel confidence scoring mechanism for deep neural networks based on a two-model paradigm involving a base model and a meta-model. The confidence score is learned by the meta-model observing the base model succeeding/failing at its task. As features to the meta-model, we investigate linear classifier probes inserted between the various layers of the base model. Our experiments demonstrate that this approach outperforms various baselines in a filtering task, i.e., task of rejecting samples with low confidence. Experimental results are presented using CIFAR-10 and CIFAR-100 dataset with and without added noise. We discuss the importance of confidence scoring to bridge the gap between experimental and real-world applications.