# Contrastive Learning enhanced Author-Style Headline Generation

arXiv: https://arxiv.org/abs/2211.03305 · HTML: https://mlsanity.com/paper/2211.03305/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 24 references that are in the map.
- First posted on arXiv 7 Nov 2022.

## Builds on (most cited first, 17 of 17)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Neural Machine Translation by Jointly Learning to Align and Translate](https://mlsanity.com/paper/1409.0473/) (2014)
- [Get To The Point: Summarization with Pointer-Generator Networks](https://mlsanity.com/paper/1704.04368/) (2017)
- [Adafactor: Adaptive Learning Rates with Sublinear Memory Cost](https://mlsanity.com/paper/1804.04235/) (2018)
- [MASS: Masked Sequence to Sequence Pre-training for Language Generation](https://mlsanity.com/paper/1905.02450/) (2019)
- [SimCLS: A Simple Framework for Contrastive Learning of Abstractive Summarization](https://mlsanity.com/paper/2106.01890/) (2021)
- [Style Transformer: Unpaired Text Style Transfer without Disentangled Latent Representation](https://mlsanity.com/paper/1905.05621/) (2019)
- [Contrastive Learning with Adversarial Perturbations for Conditional Text Generation](https://mlsanity.com/paper/2012.07280/) (2020)
- [Hooks in the Headline: Learning to Generate Headlines with Controlled Styles](https://mlsanity.com/paper/2004.01980/) (2020)
- [Weakly Supervised Contrastive Learning for Chest X-Ray Report Generation](https://mlsanity.com/paper/2109.12242/) (2021)
- [Clickbait? Sensational Headline Generation with Auto-tuned Reinforcement Learning](https://mlsanity.com/paper/1909.03582/) (2019)
- [Self-Attentive Model for Headline Generation](https://mlsanity.com/paper/1901.07786/) (2019)
- [Group-wise Contrastive Learning for Neural Dialogue Generation](https://mlsanity.com/paper/2009.07543/) (2020)
- [Diverse, Controllable, and Keyphrase-Aware: A Corpus and Method for News Multi-Headline Generation](https://mlsanity.com/paper/2004.03875/) (2020)

## Built on by (most cited first, 2 of 2)

- [Fact-Preserved Personalized News Headline Generation](https://mlsanity.com/paper/2501.11828/) (2025)
- [BeliN: A Novel Corpus for Bengali Religious News Headline Generation using Contextual Feature Fusion](https://mlsanity.com/paper/2501.01069/) (2025)

## Abstract (arXiv)

Headline generation is a task of generating an appropriate headline for a given article, which can be further used for machine-aided writing or enhancing the click-through ratio. Current works only use the article itself in the generation, but have not taken the writing style of headlines into consideration. In this paper, we propose a novel Seq2Seq model called CLH3G (Contrastive Learning enhanced Historical Headlines based Headline Generation) which can use the historical headlines of the articles that the author wrote in the past to improve the headline generation of current articles. By taking historical headlines into account, we can integrate the stylistic features of the author into our model, and generate a headline not only appropriate for the article, but also consistent with the author's style. In order to efficiently learn the stylistic features of the author, we further introduce a contrastive learning based auxiliary task for the encoder of our model. Besides, we propose two methods to use the learned stylistic features to guide both the pointer and the decoder during the generation. Experimental results show that historical headlines of the same user can improve the headline generation significantly, and both the contrastive learning module and the two style features fusion methods can further boost the performance.