# Calibrating LLMs with Information-Theoretic Evidential Deep Learning

arXiv: https://arxiv.org/abs/2502.06351 · HTML: https://mlsanity.com/paper/2502.06351/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 10 Feb 2025.

## Built on by (most cited first, 15 of 15)

- [Uncertainty Estimation by Flexible Evidential Deep Learning](https://mlsanity.com/paper/2510.18322/) (2025)
- [Detecting Misbehaviors of Large Vision-Language Models by Evidential Uncertainty Quantification](https://mlsanity.com/paper/2602.05535/) (2026)
- [Visual Instruction Bottleneck Tuning](https://mlsanity.com/paper/2505.13946/) (2025)
- [Steering Large Reasoning Models towards Concise Reasoning via Flow Matching](https://mlsanity.com/paper/2602.05539/) (2026)
- [Online Reasoning Calibration: Test-Time Training Enables Generalizable Conformal LLM Reasoning](https://mlsanity.com/paper/2604.01170/) (2026)
- [Toward Efficient Uncertainty in LLMs through Evidential Knowledge Distillation](https://mlsanity.com/paper/2507.18366/) (2025)
- [Evidential Transformation Network: Turning Pretrained Models into Evidential Models for Post-hoc Uncertainty Estimation](https://mlsanity.com/paper/2604.08627/) (2026)
- [Breaking Memorization Barriers in LLM Code Fine-Tuning via Information Bottleneck for Improved Generalization](https://mlsanity.com/paper/2510.16022/) (2025)
- [CaliDist: Calibrating Large Language Models via Behavioral Robustness to Distraction](https://mlsanity.com/paper/2606.05799/) (2026)
- [What Intermediate Layers Know: Detecting Jailbreaks from Entropy Dynamics](https://mlsanity.com/paper/2606.25182/) (2026)
- [Bayesian Sparse Low-Rank Adaptation for Large Language Model Uncertainty Estimation](https://mlsanity.com/paper/2607.02182/) (2026)
- [Possibilistic Predictive Uncertainty for Deep Learning](https://mlsanity.com/paper/2605.00600/) (2026)
- [Rethinking Vacuity for OOD Detection in Evidential Deep Learning](https://mlsanity.com/paper/2605.06382/) (2026)
- [On the Construction and Implications of Low-Loss Valleys in LoRA-based Bayesian Inference](https://mlsanity.com/paper/2605.29580/) (2026)
- [Uncertainty quantification for trustworthy deep learning: Methods and measures](https://mlsanity.com/paper/2607.28248/) (2026)

## Abstract (arXiv)

Fine-tuned large language models (LLMs) often exhibit overconfidence, particularly when trained on small datasets, resulting in poor calibration and inaccurate uncertainty estimates. Evidential Deep Learning (EDL), an uncertainty-aware approach, enables uncertainty estimation in a single forward pass, making it a promising method for calibrating fine-tuned LLMs. However, despite its computational efficiency, EDL is prone to overfitting, as its training objective can result in overly concentrated probability distributions. To mitigate this, we propose regularizing EDL by incorporating an information bottleneck (IB). Our approach IB-EDL suppresses spurious information in the evidence generated by the model and encourages truly predictive information to influence both the predictions and uncertainty estimates. Extensive experiments across various fine-tuned LLMs and tasks demonstrate that IB-EDL outperforms both existing EDL and non-EDL approaches. By improving the trustworthiness of LLMs, IB-EDL facilitates their broader adoption in domains requiring high levels of confidence calibration. Code is available at https://github.com/sandylaker/ib-edl.