# Beyond Quality: Unlocking Diversity in Ad Headline Generation with Large Language Models

arXiv: https://arxiv.org/abs/2508.18739 · HTML: https://mlsanity.com/paper/2508.18739/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 36 references that are in the map.
- First posted on arXiv 26 Aug 2025.

## Builds on (most cited first, 20 of 20)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://mlsanity.com/paper/2402.03300/) (2024)
- [Qwen2.5 Technical Report](https://mlsanity.com/paper/2412.15115/) (2024)
- [The Curious Case of Neural Text Degeneration](https://mlsanity.com/paper/1904.09751/) (2019)
- [Hierarchical Neural Story Generation](https://mlsanity.com/paper/1805.04833/) (2018)
- [A Diversity-Promoting Objective Function for Neural Conversation Models](https://mlsanity.com/paper/1510.03055/) (2015)
- [Safe RLHF: Safe Reinforcement Learning from Human Feedback](https://mlsanity.com/paper/2310.12773/) (2023)
- [Texygen: A Benchmarking Platform for Text Generation Models](https://mlsanity.com/paper/1802.01886/) (2018)
- [Fine-Grained Human Feedback Gives Better Rewards for Language Model Training](https://mlsanity.com/paper/2306.01693/) (2023)
- [Understanding the Effects of RLHF on LLM Generalisation and Diversity](https://mlsanity.com/paper/2310.06452/) (2023)
- [Making Retrieval-Augmented Language Models Robust to Irrelevant Context](https://mlsanity.com/paper/2310.01558/) (2023)
- [Evaluating the Evaluation of Diversity in Natural Language Generation](https://mlsanity.com/paper/2004.02990/) (2020)
- [LONGER: Scaling Up Long Sequence Modeling in Industrial Recommenders](https://mlsanity.com/paper/2505.04421/) (2025)
- [Dipper: Diversity in Prompts for Producing Large Language Model Ensembles in Reasoning tasks](https://mlsanity.com/paper/2412.15238/) (2024)
- [Clickbait? Sensational Headline Generation with Auto-tuned Reinforcement Learning](https://mlsanity.com/paper/1909.03582/) (2019)
- [Enhancing Uncertainty Modeling with Semantic Graph for Hallucination Detection](https://mlsanity.com/paper/2501.02020/) (2025)
- [Towards Fairness in Personalized Ads Using Impression Variance Aware Reinforcement Learning](https://mlsanity.com/paper/2306.03293/) (2023)
- [Improving Linguistic Diversity of Large Language Models with Possibility Exploration Fine-Tuning](https://mlsanity.com/paper/2412.03343/) (2024)
- [Panoramic Interests: Stylistic-Content Aware Personalized Headline Generation](https://mlsanity.com/paper/2501.11900/) (2025)

## Built on by (most cited first, 3 of 3)

- [Design Your Ad: Personalized Advertising Image and Text Generation with Unified Autoregressive Models](https://mlsanity.com/paper/2605.12138/) (2026)
- [Interactor: Agentic RL oriented Iterative Creation for Ad Description Generation in Sponsored Search](https://mlsanity.com/paper/2606.15911/) (2026)
- [Recontextualizing Famous Quotes for Brand Slogan Generation](https://mlsanity.com/paper/2602.06049/) (2026)

## Abstract (arXiv)

The generation of ad headlines plays a vital role in modern advertising, where both quality and diversity are essential to engage a broad range of audience segments. Current approaches primarily optimize language models for headline quality or click-through rates (CTR), often overlooking the need for diversity and resulting in homogeneous outputs. To address this limitation, we propose DIVER, a novel framework based on large language models (LLMs) that are jointly optimized for both diversity and quality. We first design a semantic- and stylistic-aware data generation pipeline that automatically produces high-quality training pairs with ad content and multiple diverse headlines. To achieve the goal of generating high-quality and diversified ad headlines within a single forward pass, we propose a multi-stage multi-objective optimization framework with supervised fine-tuning (SFT) and reinforcement learning (RL). Experiments on real-world industrial datasets demonstrate that DIVER effectively balances quality and diversity. Deployed on a large-scale content-sharing platform serving hundreds of millions of users, our framework improves advertiser value (ADVV) and CTR by 4.0% and 1.4%.