# MetaKP: On-Demand Keyphrase Generation

arXiv: https://arxiv.org/abs/2407.00191 · HTML: https://mlsanity.com/paper/2407.00191/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 25 of its 46 references that are in the map.
- First posted on arXiv 28 Jun 2024.

## Builds on (most cited first, 25 of 25)

- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [Self-Consistency Improves Chain of Thought Reasoning in Language Models](https://mlsanity.com/paper/2203.11171/) (2022)
- [The Flan Collection: Designing Data and Methods for Effective Instruction Tuning](https://mlsanity.com/paper/2301.13688/) (2023)
- [QMSum: A New Benchmark for Query-based Multi-domain Meeting Summarization](https://mlsanity.com/paper/2104.05938/) (2021)
- [Multi-Task Identification of Entities, Relations, and Coreference for Scientific Knowledge Graph Construction](https://mlsanity.com/paper/1808.09602/) (2018)
- [Plan-And-Write: Towards Better Automatic Storytelling](https://mlsanity.com/paper/1811.05701/) (2018)
- [Don't Hallucinate, Abstain: Identifying LLM Knowledge Gaps via Multi-LLM Collaboration](https://mlsanity.com/paper/2402.00367/) (2024)
- [GSum: A General Framework for Guided Neural Abstractive Summarization](https://mlsanity.com/paper/2010.08014/) (2020)
- [SemEval 2017 Task 10: ScienceIE - Extracting Keyphrases and Relations from Scientific Publications](https://mlsanity.com/paper/1704.02853/) (2017)
- [Deep Keyphrase Generation](https://mlsanity.com/paper/1704.06879/) (2017)
- [KPTimes: A Large-Scale Dataset for Keyphrase Generation on News Documents](https://mlsanity.com/paper/1911.12559/) (2019)
- [Exclusive Hierarchical Decoding for Deep Keyphrase Generation](https://mlsanity.com/paper/2004.08511/) (2020)
- [Unsupervised Text Generation by Learning from Search](https://mlsanity.com/paper/2007.08557/) (2020)
- [Learning Rich Representation of Keyphrases from Text](https://mlsanity.com/paper/2112.08547/) (2021)
- [Instruct and Extract: Instruction Tuning for On-Demand Information Extraction](https://mlsanity.com/paper/2310.16040/) (2023)
- [Topic-Aware Neural Keyphrase Generation for Social Media Language](https://mlsanity.com/paper/1906.03889/) (2019)
- [Keyphrase Generation for Scientific Document Retrieval](https://mlsanity.com/paper/2106.14726/) (2021)
- [Is ChatGPT A Good Keyphrase Generator? A Preliminary Study](https://mlsanity.com/paper/2303.13001/) (2023)
- [Representation Learning for Resource-Constrained Keyphrase Generation](https://mlsanity.com/paper/2203.08118/) (2022)
- [Rethinking Model Selection and Decoding for Keyphrase Generation with Pre-trained Sequence-to-Sequence Models](https://mlsanity.com/paper/2310.06374/) (2023)
- [Event Detection from Social Media for Epidemic Prediction](https://mlsanity.com/paper/2404.01679/) (2024)
- [A Large-Scale Dataset for Biomedical Keyphrase Generation](https://mlsanity.com/paper/2211.12124/) (2022)
- [A Preliminary Empirical Study on Prompt-based Unsupervised Keyphrase Extraction](https://mlsanity.com/paper/2405.16571/) (2024)
- [Deep Keyphrase Completion](https://mlsanity.com/paper/2111.01910/) (2021)

## Abstract (arXiv)

Traditional keyphrase prediction methods predict a single set of keyphrases per document, failing to cater to the diverse needs of users and downstream applications. To bridge the gap, we introduce on-demand keyphrase generation, a novel paradigm that requires keyphrases that conform to specific high-level goals or intents. For this task, we present MetaKP, a large-scale benchmark comprising four datasets, 7500 documents, and 3760 goals across news and biomedical domains with human-annotated keyphrases. Leveraging MetaKP, we design both supervised and unsupervised methods, including a multi-task fine-tuning approach and a self-consistency prompting method with large language models. The results highlight the challenges of supervised fine-tuning, whose performance is not robust to distribution shifts. By contrast, the proposed self-consistency prompting approach greatly improves the performance of large language models, enabling GPT-4o to achieve 0.548 SemF1, surpassing the performance of a fully fine-tuned BART-base model. Finally, we demonstrate the potential of our method to serve as a general NLP infrastructure, exemplified by its application in epidemic event detection from social media.