# XU-RS: Explaining Credal Width in Random-Set Language Models

arXiv: https://arxiv.org/abs/2609.37594 · HTML: https://mlsanity.com/paper/2609.37594/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 38 of its 74 references that are in the map.
- First posted on arXiv 29 Sep 2026.

## Builds on (most cited first, 30 of 38)

- [LoRA: Low-Rank Adaptation of Large Language Models](https://mlsanity.com/paper/2106.09685/) (2021)
- [Axiomatic Attribution for Deep Networks](https://mlsanity.com/paper/1703.01365/) (2017)
- [What Disease does this Patient Have? A Large-scale Open Domain Question Answering Dataset from Medical Exams](https://mlsanity.com/paper/2009.13081/) (2020)
- [Sanity Checks for Saliency Maps](https://mlsanity.com/paper/1810.03292/) (2018)
- [Aleatoric and Epistemic Uncertainty in Machine Learning: An Introduction to Concepts and Methods](https://mlsanity.com/paper/1910.09457/) (2019)
- [Towards Faithfully Interpretable NLP Systems: How should we define and evaluate faithfulness?](https://mlsanity.com/paper/2004.03685/) (2020)
- [Getting a CLUE: A Method for Explaining Uncertainty Estimates](https://mlsanity.com/paper/2006.06848/) (2020)
- [The Out-of-Distribution Problem in Explainability and Search Methods for Feature Importance Explanations](https://mlsanity.com/paper/2106.00786/) (2021)
- [Discretized Integrated Gradients for Explaining Language Models](https://mlsanity.com/paper/2108.13654/) (2021)
- [Rethinking Aleatoric and Epistemic Uncertainty](https://mlsanity.com/paper/2412.20892/) (2024)
- [Explaining Predictive Uncertainty with Information Theoretic Shapley Values](https://mlsanity.com/paper/2306.05724/) (2023)
- [Credal Learning Theory](https://mlsanity.com/paper/2402.00957/) (2024)
- [CreINNs: Credal-Set Interval Neural Networks for Uncertainty Estimation in Classification Tasks](https://mlsanity.com/paper/2401.05043/) (2024)
- [Random-Set Neural Networks (RS-NN)](https://mlsanity.com/paper/2307.05772/) (2023)
- [Epistemic Deep Learning](https://mlsanity.com/paper/2206.07609/) (2022)
- [Visions of a generalized probability theory](https://mlsanity.com/paper/1810.10341/) (2018)
- [Credal Wrapper of Model Averaging for Uncertainty Estimation in Classification](https://mlsanity.com/paper/2405.15047/) (2024)
- [A Unified Evaluation Framework for Epistemic Predictions](https://mlsanity.com/paper/2501.16912/) (2025)
- [Uncertainty measures: The big picture](https://mlsanity.com/paper/2104.06839/) (2021)
- [Attribution of Predictive Uncertainties in Classification Models](https://mlsanity.com/paper/2107.08756/) (2021)
- [Gradient-based Uncertainty Attribution for Explainable Bayesian Deep Learning](https://mlsanity.com/paper/2304.04824/) (2023)
- [Epistemic Artificial Intelligence is Essential for Machine Learning Models to Truly 'Know When They Do Not Know'](https://mlsanity.com/paper/2505.04950/) (2025)
- [Credal Ensemble Distillation for Uncertainty Quantification](https://mlsanity.com/paper/2511.13766/) (2025)
- [Credal and Interval Deep Evidential Classifications](https://mlsanity.com/paper/2512.05526/) (2025)
- [Random-Set Large Language Models](https://mlsanity.com/paper/2504.18085/) (2025)
- [Learning Credal Ensembles via Distributionally Robust Optimization](https://mlsanity.com/paper/2602.08470/) (2026)
- [Reasoning with random sets: An agenda for the future](https://mlsanity.com/paper/2401.09435/) (2023)
- [Belief likelihood function for generalised logistic regression](https://mlsanity.com/paper/1808.02560/) (2018)
- [Generalising realisability in statistical learning theory under epistemic uncertainty](https://mlsanity.com/paper/2402.14759/) (2024)
- [Uncertainty-Aware Autonomous Vehicles: Predicting the Road Ahead](https://mlsanity.com/paper/2510.22680/) (2025)

## Abstract (arXiv)

Uncertainty estimates tell us how unsure a model is, but not why. Without knowing which parts of an input influences a model's uncertainty, we cannot tell whether that uncertainty score depends on input features that are relevant for the task. We study this problem in randomset classifiers built using pretrained language models. These classifiers assign probability to individual answers and to groups of answers, producing lower and upper probabilities for each answer; The difference between these probabilities, called credal width, is used to represent epistemic uncertainty about an answer arising from limited training data. We propose XU-RS, a framework that attributes an answer's credal width to the input tokens (words or word pieces) supplied to a language model. XU-RS uses Expected Gradients (a standard feature attribution method) to estimate how input tokens contribute to credal width. The proposed framework is evaluated on a MedQA dataset using SmolLM3-3B and Llama-2-7B models, demonstrating that setting the embedding of a token ranked highly by XU-RS to zero (zero-masking) causes larger changes in credal width than zero-masking randomly selected tokens. In addition, we show that normalisation can cause other answer groups to influence an answer's width, reveal how token attribution can mask numerical errors, and provide diagnostic checks to verify whether a token ranked highly by XU-RS meaningfully explains model uncertainty.