# Unsupervised Keyphrase Extraction by Jointly Modeling Local and Global Context

arXiv: https://arxiv.org/abs/2109.07293 · HTML: https://mlsanity.com/paper/2109.07293/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 35 references that are in the map.
- First posted on arXiv 15 Sep 2021.

## Builds on (most cited first, 11 of 11)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Efficient Estimation of Word Representations in Vector Space](https://mlsanity.com/paper/1301.3781/) (2013)
- [XLNet: Generalized Autoregressive Pretraining for Language Understanding](https://mlsanity.com/paper/1906.08237/) (2019)
- [Deep contextualized word representations](https://mlsanity.com/paper/1802.05365/) (2018)
- [Bag of Tricks for Efficient Text Classification](https://mlsanity.com/paper/1607.01759/) (2016)
- [Distributed Representations of Sentences and Documents](https://mlsanity.com/paper/1405.4053/) (2014)
- [Unsupervised Learning of Sentence Embeddings using Compositional n-Gram Features](https://mlsanity.com/paper/1703.02507/) (2017)
- [Sentence Centrality Revisited for Unsupervised Summarization](https://mlsanity.com/paper/1906.03508/) (2019)
- [Simple Unsupervised Keyphrase Extraction using Sentence Embeddings](https://mlsanity.com/paper/1801.04470/) (2018)
- [Unsupervised Keyphrase Extraction with Multipartite Graphs](https://mlsanity.com/paper/1803.08721/) (2018)

## Built on by (most cited first, 17 of 17)

- [Open-world Story Generation with Structured Knowledge Enhancement: A Comprehensive Survey](https://mlsanity.com/paper/2212.04634/) (2022)
- [Retrieval-Augmented Multilingual Keyphrase Generation with Retriever-Generator Iterative Training](https://mlsanity.com/paper/2205.10471/) (2022)
- [Representation Learning for Resource-Constrained Keyphrase Generation](https://mlsanity.com/paper/2203.08118/) (2022)
- [From Statistical Methods to Deep Learning, Automatic Keyphrase Prediction: A Survey](https://mlsanity.com/paper/2305.02579/) (2023)
- [Pre-trained Language Models for Keyphrase Generation: A Thorough Empirical Study](https://mlsanity.com/paper/2212.10233/) (2022)
- [KPDrop: Improving Absent Keyphrase Generation](https://mlsanity.com/paper/2112.01476/) (2021)
- [General-to-Specific Transfer Labeling for Domain Adaptable Keyphrase Generation](https://mlsanity.com/paper/2208.09606/) (2022)
- [SimCKP: Simple Contrastive Learning of Keyphrase Representations](https://mlsanity.com/paper/2310.08221/) (2023)
- [Large Language Models as Zero-Shot Keyphrase Extractors: A Preliminary Empirical Study](https://mlsanity.com/paper/2312.15156/) (2023)
- [On Leveraging Encoder-only Pre-trained Language Models for Effective Keyphrase Generation](https://mlsanity.com/paper/2402.14052/) (2024)
- [Pre-Trained Language Models for Keyphrase Prediction: A Review](https://mlsanity.com/paper/2409.01087/) (2024)
- [Enhancing Phrase Representation by Information Bottleneck Guided Text Diffusion Process for Keyphrase Extraction](https://mlsanity.com/paper/2308.08739/) (2023)
- [Utilizing Cognitive Signals Generated during Human Reading to Enhance Keyphrase Extraction from Microblogs](https://mlsanity.com/paper/2606.26485/) (2026)
- [Unsupervised Keyphrase Extraction via Interpretable Neural Networks](https://mlsanity.com/paper/2203.07640/) (2022)
- [Unsupervised Open-domain Keyphrase Generation](https://mlsanity.com/paper/2306.10755/) (2023)
- [Unsupervised extraction of local and global keywords from a single text](https://mlsanity.com/paper/2307.14005/) (2023)
- [Attention-Seeker: Dynamic Self-Attention Scoring for Unsupervised Keyphrase Extraction](https://mlsanity.com/paper/2409.10907/) (2024)

## Abstract (arXiv)

Embedding based methods are widely used for unsupervised keyphrase extraction (UKE) tasks. Generally, these methods simply calculate similarities between phrase embeddings and document embedding, which is insufficient to capture different context for a more effective UKE model. In this paper, we propose a novel method for UKE, where local and global contexts are jointly modeled. From a global view, we calculate the similarity between a certain phrase and the whole document in the vector space as transitional embedding based models do. In terms of the local view, we first build a graph structure based on the document where phrases are regarded as vertices and the edges are similarities between vertices. Then, we proposed a new centrality computation method to capture local salient information based on the graph structure. Finally, we further combine the modeling of global and local context for ranking. We evaluate our models on three public benchmarks (Inspec, DUC 2001, SemEval 2010) and compare with existing state-of-the-art models. The results show that our model outperforms most models while generalizing better on input documents with different domains and length. Additional ablation study shows that both the local and global information is crucial for unsupervised keyphrase extraction tasks.