# Revisiting Zero-Shot Abstractive Summarization in the Era of Large Language Models from the Perspective of Position Bias

arXiv: https://arxiv.org/abs/2401.01989 · HTML: https://mlsanity.com/paper/2401.01989/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 35 references that are in the map.
- First posted on arXiv 3 Jan 2024.

## Builds on (most cited first, 17 of 17)

- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [Scaling Instruction-Finetuned Language Models](https://mlsanity.com/paper/2210.11416/) (2022)
- [Don't Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarization](https://mlsanity.com/paper/1808.08745/) (2018)
- [Get To The Point: Summarization with Pointer-Generator Networks](https://mlsanity.com/paper/1704.04368/) (2017)
- [PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive Summarization](https://mlsanity.com/paper/1912.08777/) (2019)
- [Text Summarization with Pretrained Encoders](https://mlsanity.com/paper/1908.08345/) (2019)
- [Benchmarking Large Language Models for News Summarization](https://mlsanity.com/paper/2301.13848/) (2023)
- [News Summarization and Evaluation in the Era of GPT-3](https://mlsanity.com/paper/2209.12356/) (2022)
- [How is ChatGPT's behavior changing over time?](https://mlsanity.com/paper/2307.09009/) (2023)
- [Better Zero-Shot Reasoning with Role-Play Prompting](https://mlsanity.com/paper/2308.07702/) (2023)
- [Abstractive Summarization of Reddit Posts with Multi-level Memory Networks](https://mlsanity.com/paper/1811.00783/) (2018)
- [Evaluating the Factual Consistency of Large Language Models Through News Summarization](https://mlsanity.com/paper/2211.08412/) (2022)
- [ChatGPT: A Meta-Analysis after 2.5 Months](https://mlsanity.com/paper/2302.13795/) (2023)
- [Revisiting Summarization Evaluation for Scientific Articles](https://mlsanity.com/paper/1604.00400/) (2016)
- [Countering the Effects of Lead Bias in News Summarization via Multi-Stage Training and Auxiliary Losses](https://mlsanity.com/paper/1909.04028/) (2019)
- [Robust Fair Clustering: A Novel Fairness Attack and Defense Framework](https://mlsanity.com/paper/2210.01953/) (2022)
- [Demoting the Lead Bias in News Summarization via Alternating Adversarial Learning](https://mlsanity.com/paper/2105.14241/) (2021)

## Built on by (most cited first, 28 of 28)

- [A Systematic Survey of Text Summarization: From Statistical Methods to Large Language Models](https://mlsanity.com/paper/2406.11289/) (2024)
- [Circuit Complexity Bounds for RoPE-based Transformer Architecture](https://mlsanity.com/paper/2411.07602/) (2024)
- [Multi-Layer Transformers Gradient Can be Approximated in Almost Linear Time](https://mlsanity.com/paper/2408.13233/) (2024)
- [Improving Faithfulness of Large Language Models in Summarization via Sliding Generation and Self-Consistency](https://mlsanity.com/paper/2407.21443/) (2024)
- [A Survey of Deep Long-Tail Classification Advancements](https://mlsanity.com/paper/2404.15593/) (2024)
- [Assessing LLMs for Zero-shot Abstractive Summarization Through the Lens of Relevance Paraphrasing](https://mlsanity.com/paper/2406.03993/) (2024)
- [Fairness Definitions in Language Models Explained](https://mlsanity.com/paper/2407.18454/) (2024)
- [StrucSum: Graph-Structured Reasoning for Long Document Extractive Summarization with LLMs](https://mlsanity.com/paper/2505.22950/) (2025)
- [Watching the AI Watchdogs: A Fairness and Robustness Analysis of AI Safety Moderation Classifiers](https://mlsanity.com/paper/2501.13302/) (2025)
- [What Affects the Stability of Tool Learning? An Empirical Study on the Robustness of Tool Learning Frameworks](https://mlsanity.com/paper/2407.03007/) (2024)
- [Disentangling Likes and Dislikes in Personalized Generative Explainable Recommendation](https://mlsanity.com/paper/2410.13248/) (2024)
- [Summarization of Opinionated Political Documents with Varied Perspectives](https://mlsanity.com/paper/2411.04093/) (2024)
- [RoToR: Towards More Reliable Responses for Order-Invariant Inputs](https://mlsanity.com/paper/2502.08662/) (2025)
- [DiffSampling: Enhancing Diversity and Accuracy in Neural Text Generation](https://mlsanity.com/paper/2502.14037/) (2025)
- [Can Large Language Models be Effective Online Opinion Miners?](https://mlsanity.com/paper/2505.15695/) (2025)
- [Re-ranking Using Large Language Models for Mitigating Exposure to Harmful Content on Social Media Platforms](https://mlsanity.com/paper/2501.13977/) (2025)
- [From Anger to Joy: How Nationality Personas Shape Emotion Attribution in Large Language Models](https://mlsanity.com/paper/2506.02431/) (2025)
- [Reranking-based Generation for Unbiased Perspective Summarization](https://mlsanity.com/paper/2506.15925/) (2025)
- ["Whose Side Are You On?" Estimating Ideology of Political and News Content Using Large Language Models and Few-shot Demonstration Selection](https://mlsanity.com/paper/2503.20797/) (2025)
- [Bridging the Culture Gap: A Framework for LLM-Driven Socio-Cultural Localization of Math Word Problems in Low-Resource Languages](https://mlsanity.com/paper/2508.14913/) (2025)
- [Input Order Shapes LLM Semantic Alignment in Multi-Document Summarization](https://mlsanity.com/paper/2512.02665/) (2025)
- [Token-Level Diagnosis of Sycophancy in LLMs with Attribution-Guided Steering](https://mlsanity.com/paper/2607.28906/) (2026)
- [From Ground Trust to Truth: Disparities in Offensive Language Judgments on Contemporary Korean Political Discourse](https://mlsanity.com/paper/2509.14712/) (2025)
- [DIAL-SUMMER: A Structured Evaluation Framework of Hierarchical Errors in Dialogue Summaries](https://mlsanity.com/paper/2602.08149/) (2026)
- [Enhancing Factuality through Consensus and Consistency in Summarization Using Minimum Bayes Risk Decoding](https://mlsanity.com/paper/2605.29336/) (2026)
- [Dialogue Summarization with Emotion Dynamics Using Topic- and Participant-Centric Decomposition](https://mlsanity.com/paper/2607.14769/) (2026)
- [Decoupling Generation and Selection for Budget-Constrained Faithful Summarization](https://mlsanity.com/paper/2608.03655/) (2026)
- [When Less Is Enough: Context Selection and Prompting Strategies for Bengali News Headline Generation](https://mlsanity.com/paper/2608.15879/) (2026)

## Abstract (arXiv)

We characterize and study zero-shot abstractive summarization in Large Language Models (LLMs) by measuring position bias, which we propose as a general formulation of the more restrictive lead bias phenomenon studied previously in the literature. Position bias captures the tendency of a model unfairly prioritizing information from certain parts of the input text over others, leading to undesirable behavior. Through numerous experiments on four diverse real-world datasets, we study position bias in multiple LLM models such as GPT 3.5-Turbo, Llama-2, and Dolly-v2, as well as state-of-the-art pretrained encoder-decoder abstractive summarization models such as Pegasus and BART. Our findings lead to novel insights and discussion on performance and position bias of models for zero-shot summarization tasks.