# Applying Transformer-based Text Summarization for Keyphrase Generation

arXiv: https://arxiv.org/abs/2209.03791 · HTML: https://mlsanity.com/paper/2209.03791/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 43 references that are in the map.
- First posted on arXiv 8 Sep 2022.

## Builds on (most cited first, 14 of 14)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [BERTScore: Evaluating Text Generation with BERT](https://mlsanity.com/paper/1904.09675/) (2019)
- [CTRL: A Conditional Transformer Language Model for Controllable Generation](https://mlsanity.com/paper/1909.05858/) (2019)
- [TLDR: Extreme Summarization of Scientific Documents](https://mlsanity.com/paper/2004.15011/) (2020)
- [SemEval 2017 Task 10: ScienceIE - Extracting Keyphrases and Relations from Scientific Publications](https://mlsanity.com/paper/1704.02853/) (2017)
- [Deep Keyphrase Generation](https://mlsanity.com/paper/1704.06879/) (2017)
- [A Review of Keyphrase Extraction](https://mlsanity.com/paper/1905.05044/) (2019)
- [Applying a Generic Sequence-to-Sequence Model for Simple and Effective Keyphrase Generation](https://mlsanity.com/paper/2201.05302/) (2022)
- [Keyphrase Generation: A Text Summarization Struggle](https://mlsanity.com/paper/1904.00110/) (2019)
- [Does Order Matter? An Empirical Study on Generating Multiple Keyphrases as a Sequence](https://mlsanity.com/paper/1909.03590/) (2019)

## Built on by (most cited first, 6 of 6)

- [KPEval: Towards Fine-Grained Semantic-Based Keyphrase Evaluation](https://mlsanity.com/paper/2303.15422/) (2023)
- [Core Building Blocks: Next Gen Geo Spatial GPT Application](https://mlsanity.com/paper/2310.11029/) (2023)
- [Pre-Trained Language Models for Keyphrase Prediction: A Review](https://mlsanity.com/paper/2409.01087/) (2024)
- [Plan, Generate and Complicate: Improving Low-resource Dialogue State Tracking via Easy-to-Difficult Zero-shot Data Augmentation](https://mlsanity.com/paper/2406.08860/) (2024)
- [Detecting Generated Scientific Papers using an Ensemble of Transformer Models](https://mlsanity.com/paper/2209.08283/) (2022)
- [Cross-Domain Robustness of Transformer-based Keyphrase Generation](https://mlsanity.com/paper/2312.10700/) (2023)

## Abstract (arXiv)

Keyphrases are crucial for searching and systematizing scholarly documents. Most current methods for keyphrase extraction are aimed at the extraction of the most significant words in the text. But in practice, the list of keyphrases often includes words that do not appear in the text explicitly. In this case, the list of keyphrases represents an abstractive summary of the source text. In this paper, we experiment with popular transformer-based models for abstractive text summarization using four benchmark datasets for keyphrase extraction. We compare the results obtained with the results of common unsupervised and supervised methods for keyphrase extraction. Our evaluation shows that summarization models are quite effective in generating keyphrases in the terms of the full-match F1-score and BERTScore. However, they produce a lot of words that are absent in the author's list of keyphrases, which makes summarization models ineffective in terms of ROUGE-1. We also investigate several ordering strategies to concatenate target keyphrases. The results showed that the choice of strategy affects the performance of keyphrase generation.