# SimCKP: Simple Contrastive Learning of Keyphrase Representations

arXiv: https://arxiv.org/abs/2310.08221 · HTML: https://mlsanity.com/paper/2310.08221/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 63 references that are in the map.
- First posted on arXiv 12 Oct 2023.

## Builds on (most cited first, 30 of 33)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [SimCSE: Simple Contrastive Learning of Sentence Embeddings](https://mlsanity.com/paper/2104.08821/) (2021)
- [A Contrastive Framework for Neural Text Generation](https://mlsanity.com/paper/2202.06417/) (2022)
- [SimCLS: A Simple Framework for Contrastive Learning of Abstractive Summarization](https://mlsanity.com/paper/2106.01890/) (2021)
- [Learning Dense Representations for Entity Retrieval](https://mlsanity.com/paper/1909.10506/) (2019)
- [Deep Keyphrase Generation](https://mlsanity.com/paper/1704.06879/) (2017)
- [Multi-Reward Reinforced Summarization with Saliency and Entailment](https://mlsanity.com/paper/1804.06451/) (2018)
- [RankGen: Improving Text Generation with Large Ranking Models](https://mlsanity.com/paper/2205.09726/) (2022)
- [Simple Unsupervised Keyphrase Extraction using Sentence Embeddings](https://mlsanity.com/paper/1801.04470/) (2018)
- [One Size Does Not Fit All: Generating and Evaluating Variable Number of Keyphrases](https://mlsanity.com/paper/1810.05241/) (2018)
- [Neural Keyphrase Generation via Reinforcement Learning with Adaptive Rewards](https://mlsanity.com/paper/1906.04106/) (2019)
- [One2Set: Generating Diverse Keyphrases as a Set](https://mlsanity.com/paper/2105.11134/) (2021)
- [Exclusive Hierarchical Decoding for Deep Keyphrase Generation](https://mlsanity.com/paper/2004.08511/) (2020)
- [Semi-Supervised Learning for Neural Keyphrase Generation](https://mlsanity.com/paper/1808.06773/) (2018)
- [Title-Guided Encoding for Keyphrase Generation](https://mlsanity.com/paper/1808.08575/) (2018)
- [Learning Rich Representation of Keyphrases from Text](https://mlsanity.com/paper/2112.08547/) (2021)
- [An Integrated Approach for Keyphrase Generation via Exploring the Power of Retrieval and Extraction](https://mlsanity.com/paper/1904.03454/) (2019)
- [An Empirical Study on Neural Keyphrase Generation](https://mlsanity.com/paper/2009.10229/) (2020)
- [Scientific Information Extraction with Semi-supervised Neural Tagging](https://mlsanity.com/paper/1708.06075/) (2017)
- [Select, Extract and Generate: Neural Keyphrase Generation with Layer-wise Coverage Attention](https://mlsanity.com/paper/2008.01739/) (2020)
- [MDERank: A Masked Document Embedding Rank Approach for Unsupervised Keyphrase Extraction](https://mlsanity.com/paper/2110.06651/) (2021)
- [Capturing Global Informativeness in Open Domain Keyphrase Extraction](https://mlsanity.com/paper/2004.13639/) (2020)
- [Unsupervised Keyphrase Extraction by Jointly Modeling Local and Global Context](https://mlsanity.com/paper/2109.07293/) (2021)
- [Importance Estimation from Multiple Perspectives for Keyphrase Extraction](https://mlsanity.com/paper/2110.09749/) (2021)
- [UniKeyphrase: A Unified Extraction and Generation Framework for Keyphrase Prediction](https://mlsanity.com/paper/2106.04847/) (2021)
- [Heterogeneous Graph Neural Networks for Keyphrase Generation](https://mlsanity.com/paper/2109.04703/) (2021)
- [Representation Learning for Resource-Constrained Keyphrase Generation](https://mlsanity.com/paper/2203.08118/) (2022)
- [Keyphrase Prediction With Pre-trained Language Model](https://mlsanity.com/paper/2004.10462/) (2020)

## Built on by (most cited first, 4 of 4)

- [One2set + Large Language Model: Best Partners for Keyphrase Generation](https://mlsanity.com/paper/2410.03421/) (2024)
- [QExplorer: Large Language Model Based Query Extraction for Toxic Content Exploration](https://mlsanity.com/paper/2502.18480/) (2025)
- [Zero-Shot Keyphrase Generation: Investigating Specialized Instructions and Multi-Sample Aggregation on Large Language Models](https://mlsanity.com/paper/2503.00597/) (2025)
- [An Analysis of Datasets, Metrics and Models in Keyphrase Generation](https://mlsanity.com/paper/2506.10346/) (2025)

## Abstract (arXiv)

Keyphrase generation (KG) aims to generate a set of summarizing words or phrases given a source document, while keyphrase extraction (KE) aims to identify them from the text. Because the search space is much smaller in KE, it is often combined with KG to predict keyphrases that may or may not exist in the corresponding document. However, current unified approaches adopt sequence labeling and maximization-based generation that primarily operate at a token level, falling short in observing and scoring keyphrases as a whole. In this work, we propose SimCKP, a simple contrastive learning framework that consists of two stages: 1) An extractor-generator that extracts keyphrases by learning context-aware phrase-level representations in a contrastive manner while also generating keyphrases that do not appear in the document; 2) A reranker that adapts scores for each generated phrase by likewise aligning their representations with the corresponding document. Experimental results on multiple benchmark datasets demonstrate the effectiveness of our proposed approach, which outperforms the state-of-the-art models by a significant margin.