# General-to-Specific Transfer Labeling for Domain Adaptable Keyphrase Generation

arXiv: https://arxiv.org/abs/2208.09606 · HTML: https://mlsanity.com/paper/2208.09606/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 35 of its 58 references that are in the map.
- First posted on arXiv 20 Aug 2022.

## Builds on (most cited first, 30 of 35)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [Don't Stop Pretraining: Adapt Language Models to Domains and Tasks](https://mlsanity.com/paper/2004.10964/) (2020)
- [LUKE: Deep Contextualized Entity Representations with Entity-aware Self-attention](https://mlsanity.com/paper/2010.01057/) (2020)
- [Revisiting Self-Training for Neural Sequence Generation](https://mlsanity.com/paper/1909.13788/) (2019)
- [Deep Keyphrase Generation](https://mlsanity.com/paper/1704.06879/) (2017)
- [Pretrained Encyclopedia: Weakly Supervised Knowledge-Pretrained Language Model](https://mlsanity.com/paper/1912.09637/) (2019)
- [BOND: BERT-Assisted Open-Domain Named Entity Recognition with Distant Supervision](https://mlsanity.com/paper/2006.15509/) (2020)
- [Learning Dense Representations of Phrases at Scale](https://mlsanity.com/paper/2012.12624/) (2020)
- [Learning to Self-Train for Semi-Supervised Few-Shot Classification](https://mlsanity.com/paper/1906.00562/) (2019)
- [Fine-Tuning Pre-trained Language Model with Weak Supervision: A Contrastive-Regularized Self-Training Approach](https://mlsanity.com/paper/2010.07835/) (2020)
- [One Size Does Not Fit All: Generating and Evaluating Variable Number of Keyphrases](https://mlsanity.com/paper/1810.05241/) (2018)
- [Phrase-BERT: Improved Phrase Embeddings from BERT with an Application to Corpus Exploration](https://mlsanity.com/paper/2109.06304/) (2021)
- [Neural Keyphrase Generation via Reinforcement Learning with Adaptive Rewards](https://mlsanity.com/paper/1906.04106/) (2019)
- [KPTimes: A Large-Scale Dataset for Keyphrase Generation on News Documents](https://mlsanity.com/paper/1911.12559/) (2019)
- [One2Set: Generating Diverse Keyphrases as a Set](https://mlsanity.com/paper/2105.11134/) (2021)
- [Distantly-Supervised Named Entity Recognition with Noise-Robust Learning and Language Model Augmented Self-Training](https://mlsanity.com/paper/2109.05003/) (2021)
- [Semi-Supervised Learning for Neural Keyphrase Generation](https://mlsanity.com/paper/1808.06773/) (2018)
- [Title-Guided Encoding for Keyphrase Generation](https://mlsanity.com/paper/1808.08575/) (2018)
- [Learning Rich Representation of Keyphrases from Text](https://mlsanity.com/paper/2112.08547/) (2021)
- [Topic-Aware Neural Keyphrase Generation for Social Media Language](https://mlsanity.com/paper/1906.03889/) (2019)
- [UCTopic: Unsupervised Contrastive Learning for Phrase Representations and Topic Mining](https://mlsanity.com/paper/2202.13469/) (2022)
- [Open Domain Web Keyphrase Extraction Beyond Language Modeling](https://mlsanity.com/paper/1911.02671/) (2019)
- [An Empirical Study on Neural Keyphrase Generation](https://mlsanity.com/paper/2009.10229/) (2020)
- [NER-BERT: A Pre-trained Model for Low-Resource Entity Tagging](https://mlsanity.com/paper/2112.00405/) (2021)
- [Capturing Global Informativeness in Open Domain Keyphrase Extraction](https://mlsanity.com/paper/2004.13639/) (2020)
- [Unsupervised Keyphrase Extraction by Jointly Modeling Local and Global Context](https://mlsanity.com/paper/2109.07293/) (2021)

## Built on by (most cited first, 5 of 5)

- [Unsupervised Domain Adaptation for Keyphrase Generation using Citation Contexts](https://mlsanity.com/paper/2409.13266/) (2024)
- [On Leveraging Encoder-only Pre-trained Language Models for Effective Keyphrase Generation](https://mlsanity.com/paper/2402.14052/) (2024)
- [Self-Compositional Data Augmentation for Scientific Keyphrase Generation](https://mlsanity.com/paper/2411.03039/) (2024)
- [An Analysis of Datasets, Metrics and Models in Keyphrase Generation](https://mlsanity.com/paper/2506.10346/) (2025)
- [Evaluating the Homogeneity of Keyphrase Prediction Models](https://mlsanity.com/paper/2602.12989/) (2026)

## Abstract (arXiv)

Training keyphrase generation (KPG) models require a large amount of annotated data, which can be prohibitively expensive and often limited to specific domains. In this study, we first demonstrate that large distribution shifts among different domains severely hinder the transferability of KPG models. We then propose a three-stage pipeline, which gradually guides KPG models' learning focus from general syntactical features to domain-related semantics, in a data-efficient manner. With Domain-general Phrase pre-training, we pre-train Sequence-to-Sequence models with generic phrase annotations that are widely available on the web, which enables the models to generate phrases in a wide range of domains. The resulting model is then applied in the Transfer Labeling stage to produce domain-specific pseudo keyphrases, which help adapt models to a new domain. Finally, we fine-tune the model with limited data with true labels to fully adapt it to the target domain. Our experiment results show that the proposed process can produce good-quality keyphrases in new domains and achieve consistent improvements after adaptation with limited in-domain annotated data. All code and datasets are available at https://github.com/memray/OpenNMT-kpg-release.