# Generating Representative Headlines for News Stories

arXiv: https://arxiv.org/abs/2001.09386 · HTML: https://mlsanity.com/paper/2001.09386/

## Key facts

- Cited by 39 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 26 Jan 2020.

## Built on by (most cited first, 30 of 39)

- [PRIMERA: Pyramid-based Masked Sentence Pre-training for Multi-document Summarization](https://mlsanity.com/paper/2110.08499/) (2021)
- [ConvoSumm: Conversation Summarization Benchmark and Improved Abstractive Summarization with Argument Mining](https://mlsanity.com/paper/2106.00829/) (2021)
- [PropSegmEnt: A Large-Scale Corpus for Proposition-Level Segmentation and Entailment Recognition](https://mlsanity.com/paper/2212.10750/) (2022)
- [Peek Across: Improving Multi-Document Modeling via Cross-Document Question-Answering](https://mlsanity.com/paper/2305.15387/) (2023)
- [FLUTE: A Scalable, Extensible Framework for High-Performance Federated Learning Simulations](https://mlsanity.com/paper/2203.13789/) (2022)
- [Quiz-Style Question Generation for News Stories](https://mlsanity.com/paper/2102.09094/) (2021)
- [Hallucination Detection and Hallucination Mitigation: An Investigation](https://mlsanity.com/paper/2401.08358/) (2024)
- ["Why is this misleading?": Detecting News Headline Hallucinations with Explanations](https://mlsanity.com/paper/2302.05852/) (2023)
- [From Text Segmentation to Smart Chaptering: A Novel Benchmark for Structuring Video Transcriptions](https://mlsanity.com/paper/2402.17633/) (2024)
- [CiteSum: Citation Text-guided Scientific Extreme Summarization and Domain Adaptation with Limited Supervision](https://mlsanity.com/paper/2205.06207/) (2022)
- [Harnessing the Power of LLMs: Evaluating Human-AI Text Co-Creation through the Lens of News Headline Generation](https://mlsanity.com/paper/2310.10706/) (2023)
- [AgreeSum: Agreement-Oriented Multi-Document Summarization](https://mlsanity.com/paper/2106.02278/) (2021)
- [AnswerSumm: A Manually-Curated Dataset and Pipeline for Answer Summarization](https://mlsanity.com/paper/2111.06474/) (2021)
- [MDCure: A Scalable Pipeline for Multi-Document Instruction-Following](https://mlsanity.com/paper/2410.23463/) (2024)
- [SmartBook: AI-Assisted Situation Report Generation for Intelligence Analysts](https://mlsanity.com/paper/2303.14337/) (2023)
- [XL-HeadTags: Leveraging Multimodal Retrieval Augmentation for the Multilingual Generation of News Headlines and Tags](https://mlsanity.com/paper/2406.03776/) (2024)
- [PDSum: Prototype-driven Continuous Summarization of Evolving Multi-document Sets Stream](https://mlsanity.com/paper/2302.05550/) (2023)
- [LAIT: Efficient Multi-Segment Encoding in Transformers with Layer-Adjustable Interaction](https://mlsanity.com/paper/2305.19585/) (2023)
- [NewsStories: Illustrating articles with visual summaries](https://mlsanity.com/paper/2207.13061/) (2022)
- [Multi-Document Summarization with Centroid-Based Pretraining](https://mlsanity.com/paper/2208.01006/) (2022)
- [Unsupervised Story Discovery from Continuous News Streams via Scalable Thematic Embedding](https://mlsanity.com/paper/2304.04099/) (2023)
- [SCStory: Self-supervised and Continual Online Story Discovery](https://mlsanity.com/paper/2312.03725/) (2023)
- [Multi-Perspective Abstractive Answer Summarization](https://mlsanity.com/paper/2104.08536/) (2021)
- [SumREN: Summarizing Reported Speech about Events in News](https://mlsanity.com/paper/2212.01146/) (2022)
- [SLM Meets LLM: Balancing Latency, Interpretability and Consistency in Hallucination Detection](https://mlsanity.com/paper/2408.12748/) (2024)
- [Panoramic Interests: Stylistic-Content Aware Personalized Headline Generation](https://mlsanity.com/paper/2501.11900/) (2025)
- [Extract, Denoise and Enforce: Evaluating and Improving Concept Preservation for Text-to-Text Generation](https://mlsanity.com/paper/2104.08724/) (2021)
- [Hybrid Encoder: Towards Efficient and Precise Native AdsRecommendation via Hybrid Transformer Encoding Networks](https://mlsanity.com/paper/2104.10925/) (2021)
- [A Hierarchical Encoding-Decoding Scheme for Abstractive Multi-document Summarization](https://mlsanity.com/paper/2305.08503/) (2023)
- [Let AI Entertain You: Increasing User Engagement with Generative AI and Rejection Sampling](https://mlsanity.com/paper/2312.12457/) (2023)

## Abstract (arXiv)

Millions of news articles are published online every day, which can be overwhelming for readers to follow. Grouping articles that are reporting the same event into news stories is a common way of assisting readers in their news consumption. However, it remains a challenging research problem to efficiently and effectively generate a representative headline for each story. Automatic summarization of a document set has been studied for decades, while few studies have focused on generating representative headlines for a set of articles. Unlike summaries, which aim to capture most information with least redundancy, headlines aim to capture information jointly shared by the story articles in short length, and exclude information that is too specific to each individual article. In this work, we study the problem of generating representative headlines for news stories. We develop a distant supervision approach to train large-scale generation models without any human annotation. This approach centers on two technical components. First, we propose a multi-level pre-training framework that incorporates massive unlabeled corpus with different quality-vs.-quantity balance at different levels. We show that models trained within this framework outperform those trained with pure human curated corpus. Second, we propose a novel self-voting-based article attention layer to extract salient information shared by multiple articles. We show that models that incorporate this layer are robust to potential noises in news stories and outperform existing baselines with or without noises. We can further enhance our model by incorporating human labels, and we show our distant supervision approach significantly reduces the demand on labeled data.