mlsanityData to 14 Oct 2026

Is ChatGPT A Good Keyphrase Generator? A Preliminary Study

arXiv 2303.13001 ↗First posted 23 Mar 2023cs.CLcited by 17

Not on the map. Why?

Abstract (arXiv)

The emergence of ChatGPT has recently garnered significant attention from the computational linguistics community. To demonstrate its capabilities as a keyphrase generator, we conduct a preliminary evaluation of ChatGPT for the keyphrase generation task. We evaluate its performance in various aspects, including keyphrase generation prompts, keyphrase generation diversity, and long document understanding. Our evaluation is based on six benchmark datasets, and we adopt the prompt suggested by OpenAI while extending it to six candidate prompts. We find that ChatGPT performs exceptionally well on all six candidate prompts, with minor performance differences observed across the datasets. Based on our findings, we conclude that ChatGPT has great potential for keyphrase generation. Moreover, we discover that ChatGPT still faces challenges when it comes to generating absent keyphrases. Meanwhile, in the final section, we also present some limitations and future expansions of this report.

Key facts. Cited by 17 papers in the map (data through 14 Oct 2026). Builds on 15 of its 30 references that are in the map. First posted on arXiv 23 Mar 2023.

Builds on

15 of 30 references.

References · 15

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2022Training language models to follow instructions with human feedbackcited by 15,971
  3. 2022Emergent Abilities of Large Language Modelscited by 2,512
  4. 2023A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivitycited by 894
  5. 2017Deep Keyphrase Generationcited by 102
  6. 2019Neural Keyphrase Generation via Reinforcement Learning with Adaptive Rewardscited by 41
  7. 2021One2Set: Generating Diverse Keyphrases as a Setcited by 41
  8. 2020Exclusive Hierarchical Decoding for Deep Keyphrase Generationcited by 38
  9. 2019Open Domain Web Keyphrase Extraction Beyond Language Modelingcited by 28
  10. 2020Capturing Global Informativeness in Open Domain Keyphrase Extractioncited by 17
  11. 2021Importance Estimation from Multiple Perspectives for Keyphrase Extractioncited by 16
  12. 2022WR-ONE2SET: Towards Well-Calibrated Keyphrase Generationcited by 13
3 more
  1. 2022Pre-trained Language Models for Keyphrase Generation: A Thorough Empirical Studycited by 13
  2. 2020Diverse Keyphrase Generation with Neural Unlikelihood Trainingcited by 7
  3. 2022Hyperbolic Relevance Matching for Neural Keyphrase Extractioncited by 7

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 5; 2024: 9; 2025: 2; 2026: 1.

17 papers.

Sort:

  1. 2024Leak, Cheat, Repeat: Data Contamination and Evaluation Malpractices in Closed-Source LLMscited by 229
  2. 2023Exploring the Use of Large Language Models for Reference-Free Text Quality Evaluation: An Empirical Studycited by 85
  3. 2024Counting-Stars: A Multi-evidence, Position-aware, and Scalable Benchmark for Evaluating Long-Context Large Language Modelscited by 17
  4. 2023ChatGPT vs State-of-the-Art Models: A Benchmarking Study in Keyphrase Generation Taskcited by 13
  5. 2023Enhancing In-Context Learning with Answer Feedback for Multi-Span Question Answeringcited by 5
  6. 2024From Lazy to Prolific: Tackling Missing Labels in Open Vocabulary Extreme Classification by Positive-Unlabeled Sequence Learningcited by 5
  7. 2023KPEval: Towards Fine-Grained Semantic-Based Keyphrase Evaluationcited by 4
  8. 2023Large Language Models as Zero-Shot Keyphrase Extractors: A Preliminary Empirical Studycited by 4
  9. 2024A Preliminary Empirical Study on Prompt-based Unsupervised Keyphrase Extractioncited by 3
  10. 2024Pre-Trained Language Models for Keyphrase Prediction: A Reviewcited by 3
  11. 2024One2set + Large Language Model: Best Partners for Keyphrase Generationcited by 3
  12. 2024Exploring Fine-tuned Generative Models for Keyphrase Selection: A Case Study for Russiancited by 1
5 more
  1. 2026M"OVE: A Holistic LLM Benchmark for the German Public Sectorcited by 1
  2. 2024MetaKP: On-Demand Keyphrase Generationcited by 0
  3. 2024Key Algorithms for Keyphrase Generation: Instruction-Based LLMs for Russian Scientific Keyphrasescited by 0
  4. 2025Zero-Shot Keyphrase Generation: Investigating Specialized Instructions and Multi-Sample Aggregation on Large Language Modelscited by 0
  5. 2025An Analysis of Datasets, Metrics and Models in Keyphrase Generationcited by 0