mlsanityData to 7 Oct 2026

DUD: Decoupled Update Dynamics for Reliable Uncertainty Quantification in Large Language Models

arXiv 2608.03411 ↗First posted 4 Aug 2026cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

Accurate Uncertainty Quantification (UQ) is critical for reliable deployment of Large Language Models (LLMs), yet traditional probability-based metrics often fail to capture the model's true epistemic state. While recent mechanistic approaches leverage hidden state dynamics, they typically aggregate residual stream updates, conflating the distinct roles of parametric memory (Feed-Forward Networks) and contextual processing (Attention). We argue that this aggregation obscures fine-grained mechanistic conflicts, such as memory-context misalignment, that are fundamental indicators of uncertainty. To address this, we introduce Decoupled Update Dynamics (DUD), a framework that explicitly decouples FFN and Attention contributions via noise-induced causal interventions. By quantifying the independent restoration capabilities of each module, we construct a dual-stream dynamic profile that captures the model's internal fragility. Extensive experiments demonstrate that DUD significantly outperforms state-of-the-art baselines in both uncertainty estimation and calibration, while exhibiting superior cross-dataset generalization, validating decoupled dynamics as a robust proxy for model faithfulness.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 15 of its 32 references that are in the map. First posted on arXiv 4 Aug 2026.

Builds on

15 of 32 references.

References · 15

  1. 2019BERTScore: Evaluating Text Generation with BERTcited by 5,462
  2. 2017On Calibration of Modern Neural Networkscited by 4,860
  3. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,115
  4. 2018HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answeringcited by 3,686
  5. 2017TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehensioncited by 2,895
  6. 2022Locating and Editing Factual Associations in GPTcited by 2,609
  7. 2024Gemma 2: Improving Open Language Models at a Practical Sizecited by 1,928
  8. 2020Transformer Feed-Forward Layers Are Key-Value Memoriescited by 1,307
  9. 2023The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasetscited by 673
  10. 2018Predictive Uncertainty Estimation via Prior Networkscited by 555
  11. 2024INSIDE: LLMs' Internal States Retain the Power of Hallucination Detectioncited by 317
  12. 2024Semantic Entropy Probes: Robust and Cheap Hallucination Detection in LLMscited by 246
3 more
  1. 2022Out-of-Distribution Detection and Selective Generation for Conditional Language Modelscited by 193
  2. 2024Lookback Lens: Detecting and Mitigating Contextual Hallucinations in Large Language Models Using Only Attention Mapscited by 140
  3. 2025ICR Probe: Tracking Hidden State Dynamics for Reliable Hallucination Detection in LLMscited by 31

Built on by

Not cited yet.