# Deep Reinforcement Learning with Distributional Semantic Rewards for Abstractive Summarization

arXiv: https://arxiv.org/abs/1909.00141 · HTML: https://mlsanity.com/paper/1909.00141/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 31 Aug 2019.

## Built on by (most cited first, 24 of 24)

- [Automatically Correcting Large Language Models: Surveying the landscape of diverse self-correction strategies](https://mlsanity.com/paper/2308.03188/) (2023)
- [BRIO: Bringing Order to Abstractive Summarization](https://mlsanity.com/paper/2203.16804/) (2022)
- [SimCLS: A Simple Framework for Contrastive Learning of Abstractive Summarization](https://mlsanity.com/paper/2106.01890/) (2021)
- [On Learning to Summarize with Large Language Models as References](https://mlsanity.com/paper/2305.14239/) (2023)
- [Gender Biases in Automatic Evaluation Metrics for Image Captioning](https://mlsanity.com/paper/2305.14711/) (2023)
- [BERTTune: Fine-Tuning Neural Machine Translation with BERTScore](https://mlsanity.com/paper/2106.02208/) (2021)
- [Make The Most of Prior Data: A Solution for Interactive Text Summarization with Preference Feedback](https://mlsanity.com/paper/2204.05512/) (2022)
- [Semi-Offline Reinforcement Learning for Optimized Text Generation](https://mlsanity.com/paper/2306.09712/) (2023)
- [Robust Deep Reinforcement Learning for Extractive Legal Summarization](https://mlsanity.com/paper/2111.07158/) (2021)
- [COLO: A Contrastive Learning based Re-ranking Framework for One-Stage Summarization](https://mlsanity.com/paper/2209.14569/) (2022)
- [N-Critics: Self-Refinement of Large Language Models with Ensemble of Critics](https://mlsanity.com/paper/2310.18679/) (2023)
- [SGIC: A Self-Guided Iterative Calibration Framework for RAG](https://mlsanity.com/paper/2506.16172/) (2025)
- [BEAMetrics: A Benchmark for Language Generation Evaluation Evaluation](https://mlsanity.com/paper/2110.09147/) (2021)
- [Evaluating and Improving Factuality in Multimodal Abstractive Summarization](https://mlsanity.com/paper/2211.02580/) (2022)
- [Focus-Driven Contrastive Learniang for Medical Question Summarization](https://mlsanity.com/paper/2209.00484/) (2022)
- [Inverse Reinforcement Learning for Text Summarization](https://mlsanity.com/paper/2212.09917/) (2022)
- [RewardsOfSum: Exploring Reinforcement Learning Rewards for Summarisation](https://mlsanity.com/paper/2106.04080/) (2021)
- [Few-shot Question Generation for Personalized Feedback in Intelligent Tutoring Systems](https://mlsanity.com/paper/2206.04187/) (2022)
- [Improving abstractive summarization with energy-based re-ranking](https://mlsanity.com/paper/2210.15553/) (2022)
- [An Empirical Study on Self-correcting Large Language Models for Data Science Code Generation](https://mlsanity.com/paper/2408.15658/) (2024)
- [A Multi-Document Coverage Reward for RELAXed Multi-Document Summarization](https://mlsanity.com/paper/2203.02894/) (2022)
- [Reinforcement Replaces Supervision: Query focused Summarization using Deep Reinforcement Learning](https://mlsanity.com/paper/2311.17514/) (2023)
- [SumTra: A Differentiable Pipeline for Few-Shot Cross-Lingual Summarization](https://mlsanity.com/paper/2403.13240/) (2024)
- [Reinforcement Learning without Human Feedback for Last Mile Fine-Tuning of Large Language Models](https://mlsanity.com/paper/2408.16753/) (2024)

## Abstract (arXiv)

Deep reinforcement learning (RL) has been a commonly-used strategy for the abstractive summarization task to address both the exposure bias and non-differentiable task issues. However, the conventional reward Rouge-L simply looks for exact n-grams matches between candidates and annotated references, which inevitably makes the generated sentences repetitive and incoherent. In this paper, instead of Rouge-L, we explore the practicability of utilizing the distributional semantics to measure the matching degrees. With distributional semantics, sentence-level evaluation can be obtained, and semantically-correct phrases can also be generated without being limited to the surface form of the reference sentences. Human judgments on Gigaword and CNN/Daily Mail datasets show that our proposed distributional semantics reward (DSR) has distinct superiority in capturing the lexical and compositional diversity of natural language.