# Evaluating the Homogeneity of Keyphrase Prediction Models

arXiv: https://arxiv.org/abs/2602.12989 · HTML: https://mlsanity.com/paper/2602.12989/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 24 of its 42 references that are in the map.
- First posted on arXiv 13 Feb 2026.

## Builds on (most cited first, 24 of 24)

- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [A Monolingual Approach to Contextualized Word Embeddings for Mid-Resource Languages](https://mlsanity.com/paper/2006.06202/) (2020)
- [Deep Keyphrase Generation](https://mlsanity.com/paper/1704.06879/) (2017)
- [One Size Does Not Fit All: Generating and Evaluating Variable Number of Keyphrases](https://mlsanity.com/paper/1810.05241/) (2018)
- [Unsupervised Keyphrase Extraction with Multipartite Graphs](https://mlsanity.com/paper/1803.08721/) (2018)
- [Keyphrase Generation with Correlation Constraints](https://mlsanity.com/paper/1808.07185/) (2018)
- [Neural Keyphrase Generation via Reinforcement Learning with Adaptive Rewards](https://mlsanity.com/paper/1906.04106/) (2019)
- [KPTimes: A Large-Scale Dataset for Keyphrase Generation on News Documents](https://mlsanity.com/paper/1911.12559/) (2019)
- [One2Set: Generating Diverse Keyphrases as a Set](https://mlsanity.com/paper/2105.11134/) (2021)
- [Learning Rich Representation of Keyphrases from Text](https://mlsanity.com/paper/2112.08547/) (2021)
- [An Integrated Approach for Keyphrase Generation via Exploring the Power of Retrieval and Extraction](https://mlsanity.com/paper/1904.03454/) (2019)
- [An Empirical Study on Neural Keyphrase Generation](https://mlsanity.com/paper/2009.10229/) (2020)
- [Select, Extract and Generate: Neural Keyphrase Generation with Layer-wise Coverage Attention](https://mlsanity.com/paper/2008.01739/) (2020)
- [Keyphrase Generation for Scientific Document Retrieval](https://mlsanity.com/paper/2106.14726/) (2021)
- [UniKeyphrase: A Unified Extraction and Generation Framework for Keyphrase Prediction](https://mlsanity.com/paper/2106.04847/) (2021)
- [Applying a Generic Sequence-to-Sequence Model for Simple and Effective Keyphrase Generation](https://mlsanity.com/paper/2201.05302/) (2022)
- [WR-ONE2SET: Towards Well-Calibrated Keyphrase Generation](https://mlsanity.com/paper/2211.06862/) (2022)
- [Redefining Absent Keyphrases and their Effect on Retrieval Effectiveness](https://mlsanity.com/paper/2103.12440/) (2021)
- [A Large-Scale Dataset for Biomedical Keyphrase Generation](https://mlsanity.com/paper/2211.12124/) (2022)
- [Identifying Reliable Evaluation Metrics for Scientific Text Revision](https://mlsanity.com/paper/2506.04772/) (2025)
- [General-to-Specific Transfer Labeling for Domain Adaptable Keyphrase Generation](https://mlsanity.com/paper/2208.09606/) (2022)
- [EUROPA: A Legal Multilingual Keyphrase Generation Dataset](https://mlsanity.com/paper/2403.00252/) (2024)
- [Unsupervised Domain Adaptation for Keyphrase Generation using Citation Contexts](https://mlsanity.com/paper/2409.13266/) (2024)
- [Self-Compositional Data Augmentation for Scientific Keyphrase Generation](https://mlsanity.com/paper/2411.03039/) (2024)

## Abstract (arXiv)

Keyphrases which are useful in several NLP and IR applications are either extracted from text or predicted by generative models. Contrarily to keyphrase extraction approaches, keyphrase generation models can predict keyphrases that do not appear in a document's text called `absent keyphrases`. This ability means that keyphrase generation models can associate a document to a notion that is not explicitly mentioned in its text. Intuitively, this suggests that for two documents treating the same subjects, a keyphrase generation model is more likely to be homogeneous in their indexing i.e. predict the same keyphrase for both documents, regardless of those keyphrases appearing in their respective text or not; something a keyphrase extraction model would fail to do. Yet, homogeneity of keyphrase prediction models is not covered by current benchmarks. In this work, we introduce a method to evaluate the homogeneity of keyphrase prediction models and study if absent keyphrase generation capabilities actually help the model to be more homogeneous. To our surprise, we show that keyphrase extraction methods are competitive with generative models, and that the ability to generate absent keyphrases can actually have a negative impact on homogeneity. Our data, code and prompts are available on huggingface and github.