# Understanding Neural Abstractive Summarization Models via Uncertainty

arXiv: https://arxiv.org/abs/2010.07882 · HTML: https://mlsanity.com/paper/2010.07882/

## Key facts

- Cited by 34 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Oct 2020.

## Built on by (most cited first, 30 of 34)

- [A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions](https://mlsanity.com/paper/2311.05232/) (2023)
- [On Hallucination and Predictive Uncertainty in Conditional Language Generation](https://mlsanity.com/paper/2103.15025/) (2021)
- [Hallucinated but Factual! Inspecting the Factuality of Hallucinations in Abstractive Summarization](https://mlsanity.com/paper/2109.09784/) (2021)
- [An Empirical Survey on Long Document Summarization: Datasets, Models and Metrics](https://mlsanity.com/paper/2207.00939/) (2022)
- [Mutual Information Alleviates Hallucinations in Abstractive Summarization](https://mlsanity.com/paper/2210.13210/) (2022)
- [Faithful or Extractive? On Mitigating the Faithfulness-Abstractiveness Trade-off in Abstractive Summarization](https://mlsanity.com/paper/2108.13684/) (2021)
- [Uncertainty in Natural Language Processing: Sources, Quantification, and Applications](https://mlsanity.com/paper/2306.04459/) (2023)
- [Factual Consistency Evaluation for Text Summarization via Counterfactual Estimation](https://mlsanity.com/paper/2108.13134/) (2021)
- [Towards Uncertainty-Aware Language Agent](https://mlsanity.com/paper/2401.14016/) (2024)
- [Distributed NLI: Learning to Predict Human Opinion Distributions for Language Reasoning](https://mlsanity.com/paper/2104.08676/) (2021)
- [Exploring Predictive Uncertainty and Calibration in NLP: A Study on the Impact of Method & Data Scarcity](https://mlsanity.com/paper/2210.15452/) (2022)
- [How Far are We from Robust Long Abstractive Summarization?](https://mlsanity.com/paper/2210.16732/) (2022)
- [On Uncertainty Calibration and Selective Generation in Probabilistic Neural Summarization: A Benchmark Study](https://mlsanity.com/paper/2304.08653/) (2023)
- [Don't Say What You Don't Know: Improving the Consistency of Abstractive Summarization by Constraining Beam Search](https://mlsanity.com/paper/2203.08436/) (2022)
- [Attention Temperature Matters in Abstractive Summarization Distillation](https://mlsanity.com/paper/2106.03441/) (2021)
- [HydraSum: Disentangling Stylistic Features in Text Summarization using Multi-Decoder Models](https://mlsanity.com/paper/2110.04400/) (2021)
- [Should We Trust This Summary? Bayesian Abstractive Summarization to The Rescue](https://mlsanity.com/paper/2105.10155/) (2021)
- [Do Multi-Document Summarization Models Synthesize?](https://mlsanity.com/paper/2301.13844/) (2023)
- [Dissecting Generation Modes for Abstractive Summarization Models via Ablation and Attribution](https://mlsanity.com/paper/2106.01518/) (2021)
- [Improved Beam Search for Hallucination Mitigation in Abstractive Summarization](https://mlsanity.com/paper/2212.02712/) (2022)
- [Trust It or Not: Confidence-Guided Automatic Radiology Report Generation](https://mlsanity.com/paper/2106.10887/) (2021)
- [Dynamic Attention-Guided Context Decoding for Mitigating Context Faithfulness Hallucinations in Large Language Models](https://mlsanity.com/paper/2501.01059/) (2025)
- [Massive-scale Decoding for Text Generation using Lattices](https://mlsanity.com/paper/2112.07660/) (2021)
- [Effective Controllable Bias Mitigation for Classification and Retrieval using Gate Adapters](https://mlsanity.com/paper/2401.16457/) (2024)
- [Optimizing Decoding Paths in Masked Diffusion Models by Quantifying Uncertainty](https://mlsanity.com/paper/2512.21336/) (2025)
- [To Point or Not to Point: Understanding How Abstractive Summarizers Paraphrase Text](https://mlsanity.com/paper/2106.01581/) (2021)
- [Uncertainty-Aware Text-to-Program for Question Answering on Structured Electronic Health Records](https://mlsanity.com/paper/2203.06918/) (2022)
- [Anytime-Valid Answer Sufficiency Certificates for LLM Generation via Sequential Information Lift](https://mlsanity.com/paper/2510.06478/) (2025)
- [Explaining Predictive Uncertainty by Looking Back at Model Explanations](https://mlsanity.com/paper/2201.03742/) (2022)
- [RTSUM: Relation Triple-based Interpretable Summarization with Multi-level Salience Visualization](https://mlsanity.com/paper/2310.13895/) (2023)

## Abstract (arXiv)

An advantage of seq2seq abstractive summarization models is that they generate text in a free-form manner, but this flexibility makes it difficult to interpret model behavior. In this work, we analyze summarization decoders in both blackbox and whitebox ways by studying on the entropy, or uncertainty, of the model's token-level predictions. For two strong pre-trained models, PEGASUS and BART on two summarization datasets, we find a strong correlation between low prediction entropy and where the model copies tokens rather than generating novel text. The decoder's uncertainty also connects to factors like sentence position and syntactic distance between adjacent pairs of tokens, giving a sense of what factors make a context particularly selective for the model's next output token. Finally, we study the relationship of decoder uncertainty and attention behavior to understand how attention gives rise to these observed effects in the model. We show that uncertainty is a useful perspective for analyzing summarization and text generation models more broadly.