# Investigating Hallucinations in Pruned Large Language Models for Abstractive Summarization

arXiv: https://arxiv.org/abs/2311.09335 · HTML: https://mlsanity.com/paper/2311.09335/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 61 of its 102 references that are in the map.
- First posted on arXiv 15 Nov 2023.

## Builds on (most cited first, 30 of 61)

- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://mlsanity.com/paper/2307.09288/) (2023)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [Mistral 7B](https://mlsanity.com/paper/2310.06825/) (2023)
- [Pointer Sentinel Mixture Models](https://mlsanity.com/paper/1609.07843/) (2016)
- [Learning both Weights and Connections for Efficient Neural Networks](https://mlsanity.com/paper/1506.02626/) (2015)
- [Language Models as Knowledge Bases?](https://mlsanity.com/paper/1909.01066/) (2019)
- [SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shot](https://mlsanity.com/paper/2301.00774/) (2023)
- [On Faithfulness and Factuality in Abstractive Summarization](https://mlsanity.com/paper/2005.00661/) (2020)
- [Abstractive Text Summarization Using Sequence-to-Sequence RNNs and Beyond](https://mlsanity.com/paper/1602.06023/) (2016)
- [A Simple and Effective Pruning Approach for Large Language Models](https://mlsanity.com/paper/2306.11695/) (2023)
- [Large Language Models](https://mlsanity.com/paper/2307.05782/) (2023)
- [BARTScore: Evaluating Generated Text as Text Generation](https://mlsanity.com/paper/2106.11520/) (2021)
- [SummEval: Re-evaluating Summarization Evaluation](https://mlsanity.com/paper/2007.12626/) (2020)
- [Object Hallucination in Image Captioning](https://mlsanity.com/paper/1809.02156/) (2018)
- [LLM-Pruner: On the Structural Pruning of Large Language Models](https://mlsanity.com/paper/2305.11627/) (2023)
- [Green AI](https://mlsanity.com/paper/1907.10597/) (2019)
- [Up or Down? Adaptive Rounding for Post-Training Quantization](https://mlsanity.com/paper/2004.10568/) (2020)
- [Evaluating the Factual Consistency of Abstractive Text Summarization](https://mlsanity.com/paper/1910.12840/) (2019)
- [What is the State of Neural Network Pruning?](https://mlsanity.com/paper/2003.03033/) (2020)
- [Benchmarking Large Language Models for News Summarization](https://mlsanity.com/paper/2301.13848/) (2023)
- [SummaC: Re-Visiting NLI-based Models for Inconsistency Detection in Summarization](https://mlsanity.com/paper/2111.09525/) (2021)
- [Asking and Answering Questions to Evaluate the Factual Consistency of Summaries](https://mlsanity.com/paper/2004.04228/) (2020)
- [Measuring and Improving Consistency in Pretrained Language Models](https://mlsanity.com/paper/2102.01017/) (2021)
- [Optimal Brain Compression: A Framework for Accurate Post-Training Quantization and Pruning](https://mlsanity.com/paper/2208.11580/) (2022)
- [FEQA: A Question Answering Evaluation Framework for Faithfulness Assessment in Abstractive Summarization](https://mlsanity.com/paper/2005.03754/) (2020)
- [OPT-IML: Scaling Language Model Instruction Meta Learning through the Lens of Generalization](https://mlsanity.com/paper/2212.12017/) (2022)
- [Accelerating Sparse Deep Neural Networks](https://mlsanity.com/paper/2104.08378/) (2021)
- [A Neural Conversational Model](https://mlsanity.com/paper/1506.05869/) (2015)

## Built on by (most cited first, 16 of 16)

- [A Comprehensive Survey on Process-Oriented Automatic Text Summarization with Exploration of LLM-Based Methods](https://mlsanity.com/paper/2403.02901/) (2024)
- [On the Impact of Calibration Data in Post-training Quantization and Pruning](https://mlsanity.com/paper/2311.09755/) (2023)
- [Investigating and Addressing Hallucinations of LLMs in Tasks Involving Negation](https://mlsanity.com/paper/2406.05494/) (2024)
- [Self-calibration for Language Model Quantization and Pruning](https://mlsanity.com/paper/2410.17170/) (2024)
- [CheckEmbed: Effective Verification of LLM Solutions to Open-Ended Tasks](https://mlsanity.com/paper/2406.02524/) (2024)
- [On the Limitations of Language Targeted Pruning: Investigating the Calibration Language Impact in Multilingual LLM Pruning](https://mlsanity.com/paper/2408.14398/) (2024)
- [Pruning as a Defense: Reducing Memorization in Large Language Models](https://mlsanity.com/paper/2502.15796/) (2025)
- [Dependency-Aware Semi-Structured Sparsity of GLU Variants in Large Language Models](https://mlsanity.com/paper/2405.01943/) (2024)
- [Less Is More? Examining Fairness in Pruned Large Language Models for Summarising Opinions](https://mlsanity.com/paper/2508.17610/) (2025)
- [An Empirical Study on Preference Tuning Generalization and Diversity Under Domain Shift](https://mlsanity.com/paper/2601.05882/) (2026)
- [Learning to Summarize by Learning to Quiz: Adversarial Agentic Collaboration for Long Document Summarization](https://mlsanity.com/paper/2509.20900/) (2025)
- [Enhancing Long Document Long Form Summarisation with Self-Planning](https://mlsanity.com/paper/2512.17179/) (2025)
- [On the Utility and Factual Reliability of Pruned Mixture-of-Experts Models in the Biomedical Domain](https://mlsanity.com/paper/2607.01444/) (2026)
- [ESG-Bench: Benchmarking Long-Context ESG Reports for Hallucination Mitigation](https://mlsanity.com/paper/2603.13154/) (2026)
- [Summarization is Not Dead Yet](https://mlsanity.com/paper/2606.08000/) (2026)
- [Different Teachers, Different Capabilities: Sub-1B On-Device Distillation for Structured Text Enrichment](https://mlsanity.com/paper/2607.08268/) (2026)

## Abstract (arXiv)

Despite the remarkable performance of generative large language models (LLMs) on abstractive summarization, they face two significant challenges: their considerable size and tendency to hallucinate. Hallucinations are concerning because they erode reliability and raise safety issues. Pruning is a technique that reduces model size by removing redundant weights, enabling more efficient sparse inference. Pruned models yield downstream task performance comparable to the original, making them ideal alternatives when operating on a limited budget. However, the effect that pruning has upon hallucinations in abstractive summarization with LLMs has yet to be explored. In this paper, we provide an extensive empirical study across five summarization datasets, two state-of-the-art pruning methods, and five instruction-tuned LLMs. Surprisingly, we find that hallucinations are less prevalent from pruned LLMs than the original models. Our analysis suggests that pruned models tend to depend more on the source document for summary generation. This leads to a higher lexical overlap between the generated summary and the source document, which could be a reason for the reduction in hallucination risk.