# Keyphrase Generation with Fine-Grained Evaluation-Guided Reinforcement Learning

arXiv: https://arxiv.org/abs/2104.08799 · HTML: https://mlsanity.com/paper/2104.08799/

## Key facts

- Cited by 13 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 29 references that are in the map.
- First posted on arXiv 18 Apr 2021.

## Builds on (most cited first, 16 of 16)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation](https://mlsanity.com/paper/1406.1078/) (2014)
- [Effective Approaches to Attention-based Neural Machine Translation](https://mlsanity.com/paper/1508.04025/) (2015)
- [Self-critical Sequence Training for Image Captioning](https://mlsanity.com/paper/1612.00563/) (2016)
- [Incorporating Copying Mechanism in Sequence-to-Sequence Learning](https://mlsanity.com/paper/1603.06393/) (2016)
- [Ranking Sentences for Extractive Summarization with Reinforcement Learning](https://mlsanity.com/paper/1802.08636/) (2018)
- [Modeling Coverage for Neural Machine Translation](https://mlsanity.com/paper/1601.04811/) (2016)
- [Deep Keyphrase Generation](https://mlsanity.com/paper/1704.06879/) (2017)
- [A Study of Reinforcement Learning for Neural Machine Translation](https://mlsanity.com/paper/1808.08866/) (2018)
- [One Size Does Not Fit All: Generating and Evaluating Variable Number of Keyphrases](https://mlsanity.com/paper/1810.05241/) (2018)
- [Keyphrase Generation with Correlation Constraints](https://mlsanity.com/paper/1808.07185/) (2018)
- [Neural Keyphrase Generation via Reinforcement Learning with Adaptive Rewards](https://mlsanity.com/paper/1906.04106/) (2019)
- [Exclusive Hierarchical Decoding for Deep Keyphrase Generation](https://mlsanity.com/paper/2004.08511/) (2020)
- [Title-Guided Encoding for Keyphrase Generation](https://mlsanity.com/paper/1808.08575/) (2018)
- [SenSeNet: Neural Keyphrase Generation with Document Structure](https://mlsanity.com/paper/2012.06754/) (2020)

## Built on by (most cited first, 13 of 13)

- [Retrieval-Augmented Multilingual Keyphrase Generation with Retriever-Generator Iterative Training](https://mlsanity.com/paper/2205.10471/) (2022)
- [Heterogeneous Graph Neural Networks for Keyphrase Generation](https://mlsanity.com/paper/2109.04703/) (2021)
- [From Statistical Methods to Deep Learning, Automatic Keyphrase Prediction: A Survey](https://mlsanity.com/paper/2305.02579/) (2023)
- [Applying a Generic Sequence-to-Sequence Model for Simple and Effective Keyphrase Generation](https://mlsanity.com/paper/2201.05302/) (2022)
- [Pre-trained Language Models for Keyphrase Generation: A Thorough Empirical Study](https://mlsanity.com/paper/2212.10233/) (2022)
- [Rethinking Model Selection and Decoding for Keyphrase Generation with Pre-trained Sequence-to-Sequence Models](https://mlsanity.com/paper/2310.06374/) (2023)
- [KPDrop: Improving Absent Keyphrase Generation](https://mlsanity.com/paper/2112.01476/) (2021)
- [General-to-Specific Transfer Labeling for Domain Adaptable Keyphrase Generation](https://mlsanity.com/paper/2208.09606/) (2022)
- [KPEval: Towards Fine-Grained Semantic-Based Keyphrase Evaluation](https://mlsanity.com/paper/2303.15422/) (2023)
- [On Leveraging Encoder-only Pre-trained Language Models for Effective Keyphrase Generation](https://mlsanity.com/paper/2402.14052/) (2024)
- [Plan, Generate and Complicate: Improving Low-resource Dialogue State Tracking via Easy-to-Difficult Zero-shot Data Augmentation](https://mlsanity.com/paper/2406.08860/) (2024)
- [Neural Keyphrase Generation: Analysis and Evaluation](https://mlsanity.com/paper/2304.13883/) (2023)
- [An Analysis of Datasets, Metrics and Models in Keyphrase Generation](https://mlsanity.com/paper/2506.10346/) (2025)

## Abstract (arXiv)

Aiming to generate a set of keyphrases, Keyphrase Generation (KG) is a classical task for capturing the central idea from a given document. Based on Seq2Seq models, the previous reinforcement learning framework on KG tasks utilizes the evaluation metrics to further improve the well-trained neural models. However, these KG evaluation metrics such as F1@5 and F1@M are only aware of the exact correctness of predictions on phrase-level and ignore the semantic similarities between similar predictions and targets, which inhibits the model from learning deep linguistic patterns. In response to this problem, we propose a new fine-grained evaluation metric to improve the RL framework, which considers different granularities: token-level F1 score, edit distance, duplication, and prediction quantities. On the whole, the new framework includes two reward functions: the fine-grained evaluation score and the vanilla F1 score. This framework helps the model identifying some partial match phrases which can be further optimized as the exact match ones. Experiments on KG benchmarks show that our proposed training framework outperforms the previous RL training frameworks among all evaluation scores. In addition, our method can effectively ease the synonym problem and generate a higher quality prediction. The source code is available at urlhttps://github.com/xuyige/FGRL4KG.