# Demoting the Lead Bias in News Summarization via Alternating Adversarial Learning

arXiv: https://arxiv.org/abs/2105.14241 · HTML: https://mlsanity.com/paper/2105.14241/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 23 references that are in the map.
- First posted on arXiv 29 May 2021.

## Builds on (most cited first, 20 of 20)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Don't Give Me the Details, Just the Summary! Topic-Aware Convolutional Neural Networks for Extreme Summarization](https://mlsanity.com/paper/1808.08745/) (2018)
- [Teaching Machines to Read and Comprehend](https://mlsanity.com/paper/1506.03340/) (2015)
- [A Discourse-Aware Attention Model for Abstractive Summarization of Long Documents](https://mlsanity.com/paper/1804.05685/) (2018)
- [Don't Take the Easy Way Out: Ensemble Based Methods for Avoiding Known Dataset Biases](https://mlsanity.com/paper/1909.03683/) (2019)
- [SummaRuNNer: A Recurrent Neural Network based Sequence Model for Extractive Summarization of Documents](https://mlsanity.com/paper/1611.04230/) (2016)
- [Neural Summarization by Extracting Sentences and Words](https://mlsanity.com/paper/1603.07252/) (2016)
- [Content Selection in Deep Learning Models of Summarization](https://mlsanity.com/paper/1810.12343/) (2018)
- [Extractive Summarization of Long Documents by Combining Global and Local Context](https://mlsanity.com/paper/1909.08089/) (2019)
- [Learning to Model and Ignore Dataset Bias with Mixed Capacity Ensembles](https://mlsanity.com/paper/2011.03856/) (2020)
- [Searching for Effective Neural Extractive Summarization: What Works and What's Next](https://mlsanity.com/paper/1907.03491/) (2019)
- [Earlier Isn't Always Better: Sub-aspect Analysis on Corpus and System Biases in Summarization](https://mlsanity.com/paper/1908.11723/) (2019)
- [Countering the Effects of Lead Bias in News Summarization via Multi-Stage Training and Auxiliary Losses](https://mlsanity.com/paper/1909.04028/) (2019)
- [Improving Context Modeling in Neural Topic Segmentation](https://mlsanity.com/paper/2010.03138/) (2020)
- [Systematically Exploring Redundancy Reduction in Summarizing Long Documents](https://mlsanity.com/paper/2012.00052/) (2020)
- [A Closer Look at Data Bias in Neural Extractive Summarization Models](https://mlsanity.com/paper/1909.13705/) (2019)
- [A Span Selection Model for Semantic Role Labeling](https://mlsanity.com/paper/1810.02245/) (2018)
- [Exploring Domain Shift in Extractive Text Summarization](https://mlsanity.com/paper/1908.11664/) (2019)
- [Topics to Avoid: Demoting Latent Confounds in Text Classification](https://mlsanity.com/paper/1909.00453/) (2019)
- [Do We Really Need That Many Parameters In Transformer For Extractive Summarization? Discourse Can Help !](https://mlsanity.com/paper/2012.02144/) (2020)

## Built on by (most cited first, 10 of 10)

- [Re-Examining System-Level Correlations of Automatic Summarization Evaluation Metrics](https://mlsanity.com/paper/2204.10216/) (2022)
- [Revisiting Zero-Shot Abstractive Summarization in the Era of Large Language Models from the Perspective of Position Bias](https://mlsanity.com/paper/2401.01989/) (2024)
- [Open Domain Multi-document Summarization: A Comprehensive Study of Model Brittleness under Retrieval](https://mlsanity.com/paper/2212.10526/) (2022)
- [A Suite of Generative Tasks for Multi-Level Multimodal Webpage Understanding](https://mlsanity.com/paper/2305.03668/) (2023)
- [Towards Understanding Large-Scale Discourse Structures in Pre-Trained and Fine-Tuned Language Models](https://mlsanity.com/paper/2204.04289/) (2022)
- [Understanding Position Bias Effects on Fairness in Social Multi-Document Summarization](https://mlsanity.com/paper/2405.01790/) (2024)
- [Modeling Endorsement for Multi-Document Abstractive Summarization](https://mlsanity.com/paper/2110.07844/) (2021)
- [Self-Supervised Position Debiasing for Large Language Models](https://mlsanity.com/paper/2401.01218/) (2024)
- [Evaluating Robustness of Dialogue Summarization Models in the Presence of Naturally Occurring Variations](https://mlsanity.com/paper/2311.08705/) (2023)
- [What Matters to an LLM? Behavioral and Computational Evidences from Summarization](https://mlsanity.com/paper/2602.00459/) (2026)

## Abstract (arXiv)

In news articles the lead bias is a common phenomenon that usually dominates the learning signals for neural extractive summarizers, severely limiting their performance on data with different or even no bias. In this paper, we introduce a novel technique to demote lead bias and make the summarizer focus more on the content semantics. Experiments on two news corpora with different degrees of lead bias show that our method can effectively demote the model's learned lead bias and improve its generality on out-of-distribution data, with little to no performance loss on in-distribution data.