# Clickbait? Sensational Headline Generation with Auto-tuned Reinforcement Learning

arXiv: https://arxiv.org/abs/1909.03582 · HTML: https://mlsanity.com/paper/1909.03582/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Sep 2019.

## Built on by (most cited first, 19 of 19)

- ["Why is this misleading?": Detecting News Headline Hallucinations with Explanations](https://mlsanity.com/paper/2302.05852/) (2023)
- [Harnessing the Power of LLMs: Evaluating Human-AI Text Co-Creation through the Lens of News Headline Generation](https://mlsanity.com/paper/2310.10706/) (2023)
- [Tweetorial Hooks: Generative AI Tools to Motivate Science on Social Media](https://mlsanity.com/paper/2305.12265/) (2023)
- [Clickbait Spoiling via Question Answering and Passage Retrieval](https://mlsanity.com/paper/2203.10282/) (2022)
- [XL-HeadTags: Leveraging Multimodal Retrieval Augmentation for the Multilingual Generation of News Headlines and Tags](https://mlsanity.com/paper/2406.03776/) (2024)
- [Nutri-bullets: Summarizing Health Studies by Composing Segments](https://mlsanity.com/paper/2103.11921/) (2021)
- [Every picture tells a story: Image-grounded controllable stylistic story generation](https://mlsanity.com/paper/2209.01638/) (2022)
- [Audience-Centric Natural Language Generation via Style Infusion](https://mlsanity.com/paper/2301.10283/) (2023)
- [HonestBait: Forward References for Attractive but Faithful Headline Generation](https://mlsanity.com/paper/2306.14828/) (2023)
- [Beyond Quality: Unlocking Diversity in Ad Headline Generation with Large Language Models](https://mlsanity.com/paper/2508.18739/) (2025)
- [Contrastive Learning enhanced Author-Style Headline Generation](https://mlsanity.com/paper/2211.03305/) (2022)
- [Generating Attractive and Authentic Copywriting from Customer Reviews](https://mlsanity.com/paper/2404.13906/) (2024)
- [Fact-Preserved Personalized News Headline Generation](https://mlsanity.com/paper/2501.11828/) (2025)
- [Enhancing Emotion Prediction in News Headlines: Insights from ChatGPT and Seq2Seq Models for Free-Text Generation](https://mlsanity.com/paper/2407.10091/) (2024)
- [Ad Headline Generation using Self-Critical Masked Language Model](https://mlsanity.com/paper/2607.06818/) (2026)
- [COBART: Controlled, Optimized, Bidirectional and Auto-Regressive Transformer for Ad Headline Generation](https://mlsanity.com/paper/2607.08071/) (2026)
- [Taming the Beast: Learning to Control Neural Conversational Models](https://mlsanity.com/paper/2108.10561/) (2021)
- [Clickbait Classification and Spoiling Using Natural Language Processing](https://mlsanity.com/paper/2306.14907/) (2023)
- [Modeling Unified Semantic Discourse Structure for High-quality Headline Generation](https://mlsanity.com/paper/2403.15776/) (2024)

## Abstract (arXiv)

Sensational headlines are headlines that capture people's attention and generate reader interest. Conventional abstractive headline generation methods, unlike human writers, do not optimize for maximal reader attention. In this paper, we propose a model that generates sensational headlines without labeled data. We first train a sensationalism scorer by classifying online headlines with many comments ("clickbait") against a baseline of headlines generated from a summarization model. The score from the sensationalism scorer is used as the reward for a reinforcement learner. However, maximizing the noisy sensationalism reward will generate unnatural phrases instead of sensational headlines. To effectively leverage this noisy reward, we propose a novel loss function, Auto-tuned Reinforcement Learning (ARL), to dynamically balance reinforcement learning (RL) with maximum likelihood estimation (MLE). Human evaluation shows that 60.8% of samples generated by our model are sensational, which is significantly better than the Pointer-Gen baseline and other RL models.