mlsanityData to 14 Oct 2026

An Analysis of Datasets, Metrics and Models in Keyphrase Generation

arXiv 2506.10346 ↗First posted 12 Jun 2025cs.IR cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

Keyphrase generation refers to the task of producing a set of words or phrases that summarises the content of a document. Continuous efforts have been dedicated to this task over the past few years, spreading across multiple lines of research, such as model architectures, data resources, and use-case scenarios. Yet, the current state of keyphrase generation remains unknown as there has been no attempt to review and analyse previous work. In this paper, we bridge this gap by presenting an analysis of over 50 research papers on keyphrase generation, offering a comprehensive overview of recent progress, limitations, and open challenges. Our findings highlight several critical issues in current evaluation practices, such as the concerning similarity among commonly-used benchmark datasets and inconsistencies in metric calculations leading to overestimated performances. Additionally, we address the limited availability of pre-trained models by releasing a strong PLM-based model for keyphrase generation as an effort to facilitate future research.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 46 of its 100 references that are in the map. First posted on arXiv 12 Jun 2025.

Builds on

46 of 100 references.

References · 46

  1. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,893
  2. 2014Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translationcited by 4,057
  3. 2014Sequence to Sequence Learning with Neural Networkscited by 2,986
  4. 2016Incorporating Copying Mechanism in Sequence-to-Sequence Learningcited by 218
  5. 2017Deep Keyphrase Generationcited by 102
  6. 2024Pretraining Data Detection for Large Language Models: A Divergence-based Calibration Methodcited by 55
  7. 2018One Size Does Not Fit All: Generating and Evaluating Variable Number of Keyphrasescited by 52
  8. 2018Keyphrase Generation with Correlation Constraintscited by 45
  9. 2019Neural Keyphrase Generation via Reinforcement Learning with Adaptive Rewardscited by 41
  10. 2019KPTimes: A Large-Scale Dataset for Keyphrase Generation on News Documentscited by 41
  11. 2021One2Set: Generating Diverse Keyphrases as a Setcited by 41
  12. 2020Exclusive Hierarchical Decoding for Deep Keyphrase Generationcited by 38
34 more
  1. 2018Semi-Supervised Learning for Neural Keyphrase Generationcited by 36
  2. 2018Title-Guided Encoding for Keyphrase Generationcited by 36
  3. 2021Learning Rich Representation of Keyphrases from Textcited by 36
  4. 2019A Review of Keyphrase Extractioncited by 33
  5. 2019Topic-Aware Neural Keyphrase Generation for Social Media Languagecited by 33
  6. 2019An Integrated Approach for Keyphrase Generation via Exploring the Power of Retrieval and Extractioncited by 28
  7. 2020An Empirical Study on Neural Keyphrase Generationcited by 25
  8. 2020Select, Extract and Generate: Neural Keyphrase Generation with Layer-wise Coverage Attentioncited by 24
  9. 2021Keyphrase Generation for Scientific Document Retrievalcited by 23
  10. 2023Is ChatGPT A Good Keyphrase Generator? A Preliminary Studycited by 17
  11. 2022Retrieval-Augmented Multilingual Keyphrase Generation with Retriever-Generator Iterative Trainingcited by 16
  12. 2021UniKeyphrase: A Unified Extraction and Generation Framework for Keyphrase Predictioncited by 15
  13. 2021Heterogeneous Graph Neural Networks for Keyphrase Generationcited by 15
  14. 2022Representation Learning for Resource-Constrained Keyphrase Generationcited by 14
  15. 2023From Statistical Methods to Deep Learning, Automatic Keyphrase Prediction: A Surveycited by 14
  16. 2021Keyphrase Generation with Fine-Grained Evaluation-Guided Reinforcement Learningcited by 13
  17. 2022LDKP: A Dataset for Identifying Keyphrases from Long Scientific Documentscited by 13
  18. 2022WR-ONE2SET: Towards Well-Calibrated Keyphrase Generationcited by 13
  19. 2023ChatGPT vs State-of-the-Art Models: A Benchmarking Study in Keyphrase Generation Taskcited by 13
  20. 2019Keyphrase Generation: A Text Summarization Strugglecited by 12
  21. 2021Keyphrase Generation Beyond the Boundaries of Title and Abstractcited by 12
  22. 2021Redefining Absent Keyphrases and their Effect on Retrieval Effectivenesscited by 11
  23. 2020TAN-NTM: Topic Attention Networks for Neural Topic Modelingcited by 10
  24. 2019Keyphrase Generation: A Multi-Aspect Surveycited by 8
  25. 2023Data Augmentation for Low-Resource Keyphrase Generationcited by 8
  26. 2023Rethinking Model Selection and Decoding for Keyphrase Generation with Pre-trained Sequence-to-Sequence Modelscited by 8
  27. 2020Diverse Keyphrase Generation with Neural Unlikelihood Trainingcited by 7
  28. 2022A Large-Scale Dataset for Biomedical Keyphrase Generationcited by 6
  29. 2022General-to-Specific Transfer Labeling for Domain Adaptable Keyphrase Generationcited by 5
  30. 2021SGG: Learning to Select, Guide, and Generate for Keyphrase Generationcited by 4
  31. 2023SimCKP: Simple Contrastive Learning of Keyphrase Representationscited by 4
  32. 2024Unsupervised Domain Adaptation for Keyphrase Generation using Citation Contextscited by 4
  33. 2024On Leveraging Encoder-only Pre-trained Language Models for Effective Keyphrase Generationcited by 3
  34. 2024One2set + Large Language Model: Best Partners for Keyphrase Generationcited by 3

Built on by

Not cited yet.