mlsanityData to 14 Oct 2026

Large Language Models as Zero-Shot Keyphrase Extractors: A Preliminary Empirical Study

arXiv 2312.15156 ↗First posted 23 Dec 2023cs.CLcited by 4

Not on the map. Why?

Abstract (arXiv)

Zero-shot keyphrase extraction aims to build a keyphrase extractor without training by human-annotated data, which is challenging due to the limited human intervention involved. Challenging but worthwhile, zero-shot setting efficiently reduces the time and effort that data labeling takes. Recent efforts on pre-trained large language models (e.g., ChatGPT and ChatGLM) show promising performance on zero-shot settings, thus inspiring us to explore prompt-based methods. In this paper, we ask whether strong keyphrase extraction models can be constructed by directly prompting the large language model ChatGPT. Through experimental results, it is found that ChatGPT still has a lot of room for improvement in the keyphrase extraction task compared to existing state-of-the-art unsupervised and supervised models.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 19 of its 45 references that are in the map. First posted on arXiv 23 Dec 2023.

Builds on

19 of 45 references.

References · 19

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2022Training language models to follow instructions with human feedbackcited by 15,971
  4. 2023ChatIE: Zero-Shot Information Extraction via Chatting with ChatGPTcited by 250
  5. 2022Large Language Models are Few-Shot Clinical Information Extractorscited by 207
  6. 2018Simple Unsupervised Keyphrase Extraction using Sentence Embeddingscited by 62
  7. 2018Unsupervised Keyphrase Extraction with Multipartite Graphscited by 50
  8. 2019Open Domain Web Keyphrase Extraction Beyond Language Modelingcited by 28
  9. 2021MDERank: A Masked Document Embedding Rank Approach for Unsupervised Keyphrase Extractioncited by 20
  10. 2020Capturing Global Informativeness in Open Domain Keyphrase Extractioncited by 17
  11. 2021Unsupervised Keyphrase Extraction by Jointly Modeling Local and Global Contextcited by 17
  12. 2023Is ChatGPT A Good Keyphrase Generator? A Preliminary Studycited by 17
7 more
  1. 2023Incorporating Distributions of Discourse Structure for Long Document Abstractive Summarizationcited by 17
  2. 2021Importance Estimation from Multiple Perspectives for Keyphrase Extractioncited by 16
  3. 2022LDKP: A Dataset for Identifying Keyphrases from Long Scientific Documentscited by 13
  4. 2023Can Query Expansion Improve Generalization of Strong Cross-Encoder Rankers?cited by 12
  5. 2023Zero-shot audio captioning with audio-language model guidance and audio context keywordscited by 11
  6. 2022KALM: Knowledge-Aware Integration of Local, Document, and Global Contexts for Long Document Understandingcited by 9
  7. 2023HiPool: Modeling Long Documents Using Graph Neural Networkscited by 2

Built on by

Cited by year

Citing papers per year: 2024: 3; 2025: 1.

4 papers.

  1. 2024LLM Reading Tea Leaves: Automatically Evaluating Topic Models with Large Language Modelscited by 11
  2. 2024From Lazy to Prolific: Tackling Missing Labels in Open Vocabulary Extreme Classification by Positive-Unlabeled Sequence Learningcited by 5
  3. 2024Key Algorithms for Keyphrase Generation: Instruction-Based LLMs for Russian Scientific Keyphrasescited by 0
  4. 2025Zero-Shot Keyphrase Generation: Investigating Specialized Instructions and Multi-Sample Aggregation on Large Language Modelscited by 0