# Cross-Domain Robustness of Transformer-based Keyphrase Generation

arXiv: https://arxiv.org/abs/2312.10700 · HTML: https://mlsanity.com/paper/2312.10700/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 21 of its 53 references that are in the map.
- First posted on arXiv 17 Dec 2023.

## Builds on (most cited first, 21 of 21)

- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive Summarization](https://mlsanity.com/paper/1912.08777/) (2019)
- [TLDR: Extreme Summarization of Scientific Documents](https://mlsanity.com/paper/2004.15011/) (2020)
- [Deep Keyphrase Generation](https://mlsanity.com/paper/1704.06879/) (2017)
- [Adapt or Get Left Behind: Domain Adaptation through BERT Language Model Finetuning for Aspect-Target Sentiment Classification](https://mlsanity.com/paper/1908.11860/) (2019)
- [Generating Scientific Claims for Zero-Shot Scientific Fact Checking](https://mlsanity.com/paper/2203.12990/) (2022)
- [Neural Keyphrase Generation via Reinforcement Learning with Adaptive Rewards](https://mlsanity.com/paper/1906.04106/) (2019)
- [KPTimes: A Large-Scale Dataset for Keyphrase Generation on News Documents](https://mlsanity.com/paper/1911.12559/) (2019)
- [Exclusive Hierarchical Decoding for Deep Keyphrase Generation](https://mlsanity.com/paper/2004.08511/) (2020)
- [Semi-Supervised Learning for Neural Keyphrase Generation](https://mlsanity.com/paper/1808.06773/) (2018)
- [Learning Rich Representation of Keyphrases from Text](https://mlsanity.com/paper/2112.08547/) (2021)
- [Meta-Transfer Learning for Low-Resource Abstractive Summarization](https://mlsanity.com/paper/2102.09397/) (2021)
- [Applying a Generic Sequence-to-Sequence Model for Simple and Effective Keyphrase Generation](https://mlsanity.com/paper/2201.05302/) (2022)
- [Pre-trained Language Models for Keyphrase Generation: A Thorough Empirical Study](https://mlsanity.com/paper/2212.10233/) (2022)
- [Keyphrase Generation: A Text Summarization Struggle](https://mlsanity.com/paper/1904.00110/) (2019)
- [Two Huge Title and Keyword Generation Corpora of Research Articles](https://mlsanity.com/paper/2002.04689/) (2020)
- [Advances of Transformer-Based Models for News Headline Generation](https://mlsanity.com/paper/2007.05044/) (2020)
- [Applying Transformer-based Text Summarization for Keyphrase Generation](https://mlsanity.com/paper/2209.03791/) (2022)
- [Automatic Summarization of Russian Texts: Comparison of Extractive and Abstractive Methods](https://mlsanity.com/paper/2206.09253/) (2022)

## Abstract (arXiv)

Modern models for text generation show state-of-the-art results in many natural language processing tasks. In this work, we explore the effectiveness of abstractive text summarization models for keyphrase selection. A list of keyphrases is an important element of a text in databases and repositories of electronic documents. In our experiments, abstractive text summarization models fine-tuned for keyphrase generation show quite high results for a target text corpus. However, in most cases, the zero-shot performance on other corpora and domains is significantly lower. We investigate cross-domain limitations of abstractive text summarization models for keyphrase generation. We present an evaluation of the fine-tuned BART models for the keyphrase selection task across six benchmark corpora for keyphrase extraction including scientific texts from two domains and news texts. We explore the role of transfer learning between different domains to improve the BART model performance on small text corpora. Our experiments show that preliminary fine-tuning on out-of-domain corpora can be effective under conditions of a limited number of samples.