mlsanityData to 14 Oct 2026

ChatGPT vs State-of-the-Art Models: A Benchmarking Study in Keyphrase Generation Task

arXiv 2304.14177 ↗First posted 27 Apr 2023cs.CL cs.AIcited by 13

Not on the map. Why?

Abstract (arXiv)

Transformer-based language models, including ChatGPT, have demonstrated exceptional performance in various natural language generation tasks. However, there has been limited research evaluating ChatGPT's keyphrase generation ability, which involves identifying informative phrases that accurately reflect a document's content. This study seeks to address this gap by comparing ChatGPT's keyphrase generation performance with state-of-the-art models, while also testing its potential as a solution for two significant challenges in the field: domain adaptation and keyphrase generation from long documents. We conducted experiments on six publicly available datasets from scientific articles and news domains, analyzing performance on both short and long documents. Our results show that ChatGPT outperforms current state-of-the-art models in all tested datasets and environments, generating high-quality keyphrases that adapt well to diverse domains and document lengths.

Key facts. Cited by 13 papers in the map (data through 14 Oct 2026). Builds on 43 of its 81 references that are in the map. First posted on arXiv 27 Apr 2023.

Builds on

43 of 81 references.

References · 43

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2020Language Models are Few-Shot Learnerscited by 32,419
  4. 2022Training language models to follow instructions with human feedbackcited by 15,971
  5. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  6. 2019BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehensioncited by 5,893
  7. 2013Efficient Estimation of Word Representations in Vector Spacecited by 4,375
  8. 2017Deep reinforcement learning from human preferencescited by 3,912
  9. 2020Learning to summarize from human feedbackcited by 2,588
  10. 2019Fine-Tuning Language Models from Human Preferencescited by 2,150
  11. 2019Unified Language Model Pre-training for Natural Language Understanding and Generationcited by 670
  12. 2023How Good Are GPT Models at Machine Translation? A Comprehensive Evaluationcited by 348
31 more
  1. 2018Reward learning from human preferences and demonstrations in Ataricited by 316
  2. 2021Recursively Summarizing Books with Human Feedbackcited by 290
  3. 2019Way Off-Policy Batch Deep Reinforcement Learning of Implicit Human Preferences in Dialogcited by 284
  4. 2018The Natural Language Decathlon: Multitask Learning as Question Answeringcited by 236
  5. 2017SemEval 2017 Task 10: ScienceIE - Extracting Keyphrases and Relations from Scientific Publicationscited by 102
  6. 2017Deep Keyphrase Generationcited by 102
  7. 2018Can Neural Machine Translation be Improved with User Feedback?cited by 72
  8. 2018Simple Unsupervised Keyphrase Extraction using Sentence Embeddingscited by 62
  9. 2018One Size Does Not Fit All: Generating and Evaluating Variable Number of Keyphrasescited by 52
  10. 2018Keyphrase Generation with Correlation Constraintscited by 45
  11. 2019Neural Keyphrase Generation via Reinforcement Learning with Adaptive Rewardscited by 41
  12. 2019KPTimes: A Large-Scale Dataset for Keyphrase Generation on News Documentscited by 41
  13. 2021One2Set: Generating Diverse Keyphrases as a Setcited by 41
  14. 2020Exclusive Hierarchical Decoding for Deep Keyphrase Generationcited by 38
  15. 2018Semi-Supervised Learning for Neural Keyphrase Generationcited by 36
  16. 2018Title-Guided Encoding for Keyphrase Generationcited by 36
  17. 2021Learning Rich Representation of Keyphrases from Textcited by 36
  18. 2019Topic-Aware Neural Keyphrase Generation for Social Media Languagecited by 33
  19. 2018Improving a Neural Semantic Parser by Counterfactual Learning from Human Bandit Feedbackcited by 32
  20. 2019An Integrated Approach for Keyphrase Generation via Exploring the Power of Retrieval and Extractioncited by 28
  21. 2020Learning to Compare for Better Training and Evaluation of Open Domain Natural Language Generation Modelscited by 28
  22. 2019Finding Generalizable Evidence by Learning to Convince Q&A Modelscited by 25
  23. 2020Select, Extract and Generate: Neural Keyphrase Generation with Layer-wise Coverage Attentioncited by 24
  24. 2023Is ChatGPT A Good Keyphrase Generator? A Preliminary Studycited by 17
  25. 2020TNT-KID: Transformer-based Neural Tagger for Keyword Identificationcited by 15
  26. 2021UniKeyphrase: A Unified Extraction and Generation Framework for Keyphrase Predictioncited by 15
  27. 2022LDKP: A Dataset for Identifying Keyphrases from Long Scientific Documentscited by 13
  28. 2021Keyphrase Generation Beyond the Boundaries of Title and Abstractcited by 12
  29. 2022A Large-Scale Dataset for Biomedical Keyphrase Generationcited by 6
  30. 2021SGG: Learning to Select, Guide, and Generate for Keyphrase Generationcited by 4
  31. 2022Query-Based Keyphrase Extraction from Long Documentscited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 2; 2024: 5; 2025: 3; 2026: 3.

13 papers.

Sort:

By area:

  1. 2023Enhancing Keyphrase Generation by BART Finetuning with Splitting and Shufflingcited by 3
  2. 2024Pre-Trained Language Models for Keyphrase Prediction: A Reviewcited by 3
  3. 2024One2set + Large Language Model: Best Partners for Keyphrase Generationcited by 3
  4. 2023Enhancing Phrase Representation by Information Bottleneck Guided Text Diffusion Process for Keyphrase Extractioncited by 2
  5. 2024Exploring Fine-tuned Generative Models for Keyphrase Selection: A Case Study for Russiancited by 1
  6. 2026M"OVE: A Holistic LLM Benchmark for the German Public Sectorcited by 1
  7. 2024Key Algorithms for Keyphrase Generation: Instruction-Based LLMs for Russian Scientific Keyphrasescited by 0
  8. 2024SEKE: Specialised Experts for Keyword Extractioncited by 0
  9. 2025Zero-Shot Keyphrase Generation: Investigating Specialized Instructions and Multi-Sample Aggregation on Large Language Modelscited by 0
  10. 2025An Analysis of Datasets, Metrics and Models in Keyphrase Generationcited by 0
  11. 2025Generative AI-Driven High-Fidelity Human Motion Simulationcited by 0
  12. 2026Attention Expansion: Enhancing Keyphrase Extraction from Long Documents with Attention-Augmented Contextualized Embeddingscited by 0
1 more
  1. 2026Semantic Networks as Clues: A Theoretical Foundation and Process Optimization for Semantic Network Constructioncited by 0