# An Analysis of Datasets, Metrics and Models in Keyphrase Generation

arXiv: https://arxiv.org/abs/2506.10346 · HTML: https://mlsanity.com/paper/2506.10346/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 46 of its 100 references that are in the map.
- First posted on arXiv 12 Jun 2025.

## Builds on (most cited first, 30 of 46)

- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation](https://mlsanity.com/paper/1406.1078/) (2014)
- [Sequence to Sequence Learning with Neural Networks](https://mlsanity.com/paper/1409.3215/) (2014)
- [Incorporating Copying Mechanism in Sequence-to-Sequence Learning](https://mlsanity.com/paper/1603.06393/) (2016)
- [Deep Keyphrase Generation](https://mlsanity.com/paper/1704.06879/) (2017)
- [Pretraining Data Detection for Large Language Models: A Divergence-based Calibration Method](https://mlsanity.com/paper/2409.14781/) (2024)
- [One Size Does Not Fit All: Generating and Evaluating Variable Number of Keyphrases](https://mlsanity.com/paper/1810.05241/) (2018)
- [Keyphrase Generation with Correlation Constraints](https://mlsanity.com/paper/1808.07185/) (2018)
- [Neural Keyphrase Generation via Reinforcement Learning with Adaptive Rewards](https://mlsanity.com/paper/1906.04106/) (2019)
- [KPTimes: A Large-Scale Dataset for Keyphrase Generation on News Documents](https://mlsanity.com/paper/1911.12559/) (2019)
- [One2Set: Generating Diverse Keyphrases as a Set](https://mlsanity.com/paper/2105.11134/) (2021)
- [Exclusive Hierarchical Decoding for Deep Keyphrase Generation](https://mlsanity.com/paper/2004.08511/) (2020)
- [Semi-Supervised Learning for Neural Keyphrase Generation](https://mlsanity.com/paper/1808.06773/) (2018)
- [Title-Guided Encoding for Keyphrase Generation](https://mlsanity.com/paper/1808.08575/) (2018)
- [Learning Rich Representation of Keyphrases from Text](https://mlsanity.com/paper/2112.08547/) (2021)
- [A Review of Keyphrase Extraction](https://mlsanity.com/paper/1905.05044/) (2019)
- [Topic-Aware Neural Keyphrase Generation for Social Media Language](https://mlsanity.com/paper/1906.03889/) (2019)
- [An Integrated Approach for Keyphrase Generation via Exploring the Power of Retrieval and Extraction](https://mlsanity.com/paper/1904.03454/) (2019)
- [An Empirical Study on Neural Keyphrase Generation](https://mlsanity.com/paper/2009.10229/) (2020)
- [Select, Extract and Generate: Neural Keyphrase Generation with Layer-wise Coverage Attention](https://mlsanity.com/paper/2008.01739/) (2020)
- [Keyphrase Generation for Scientific Document Retrieval](https://mlsanity.com/paper/2106.14726/) (2021)
- [Is ChatGPT A Good Keyphrase Generator? A Preliminary Study](https://mlsanity.com/paper/2303.13001/) (2023)
- [Retrieval-Augmented Multilingual Keyphrase Generation with Retriever-Generator Iterative Training](https://mlsanity.com/paper/2205.10471/) (2022)
- [UniKeyphrase: A Unified Extraction and Generation Framework for Keyphrase Prediction](https://mlsanity.com/paper/2106.04847/) (2021)
- [Heterogeneous Graph Neural Networks for Keyphrase Generation](https://mlsanity.com/paper/2109.04703/) (2021)
- [Representation Learning for Resource-Constrained Keyphrase Generation](https://mlsanity.com/paper/2203.08118/) (2022)
- [From Statistical Methods to Deep Learning, Automatic Keyphrase Prediction: A Survey](https://mlsanity.com/paper/2305.02579/) (2023)
- [Keyphrase Generation with Fine-Grained Evaluation-Guided Reinforcement Learning](https://mlsanity.com/paper/2104.08799/) (2021)
- [LDKP: A Dataset for Identifying Keyphrases from Long Scientific Documents](https://mlsanity.com/paper/2203.15349/) (2022)
- [WR-ONE2SET: Towards Well-Calibrated Keyphrase Generation](https://mlsanity.com/paper/2211.06862/) (2022)

## Abstract (arXiv)

Keyphrase generation refers to the task of producing a set of words or phrases that summarises the content of a document. Continuous efforts have been dedicated to this task over the past few years, spreading across multiple lines of research, such as model architectures, data resources, and use-case scenarios. Yet, the current state of keyphrase generation remains unknown as there has been no attempt to review and analyse previous work. In this paper, we bridge this gap by presenting an analysis of over 50 research papers on keyphrase generation, offering a comprehensive overview of recent progress, limitations, and open challenges. Our findings highlight several critical issues in current evaluation practices, such as the concerning similarity among commonly-used benchmark datasets and inconsistencies in metric calculations leading to overestimated performances. Additionally, we address the limited availability of pre-trained models by releasing a strong PLM-based model for keyphrase generation as an effort to facilitate future research.