# Empowering NLG: Offline Reinforcement Learning for Informal Summarization in Online Domains

arXiv: https://arxiv.org/abs/2306.17174 · HTML: https://mlsanity.com/paper/2306.17174/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 30 of its 57 references that are in the map.
- First posted on arXiv 17 Jun 2023.

## Builds on (most cited first, 30 of 30)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [RoBERTa: A Robustly Optimized BERT Pretraining Approach](https://mlsanity.com/paper/1907.11692/) (2019)
- [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://mlsanity.com/paper/2005.11401/) (2020)
- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://mlsanity.com/paper/1910.13461/) (2019)
- [Sequence to Sequence Learning with Neural Networks](https://mlsanity.com/paper/1409.3215/) (2014)
- [Addressing Function Approximation Error in Actor-Critic Methods](https://mlsanity.com/paper/1802.09477/) (2018)
- [Off-Policy Deep Reinforcement Learning without Exploration](https://mlsanity.com/paper/1812.02900/) (2018)
- [Soft Actor-Critic Algorithms and Applications](https://mlsanity.com/paper/1812.05905/) (2018)
- [Diffusion-LM Improves Controllable Text Generation](https://mlsanity.com/paper/2205.14217/) (2022)
- [Stabilizing Off-Policy Q-Learning via Bootstrapping Error Reduction](https://mlsanity.com/paper/1906.00949/) (2019)
- [Unified Language Model Pre-training for Natural Language Understanding and Generation](https://mlsanity.com/paper/1905.03197/) (2019)
- [Attention-Based Models for Speech Recognition](https://mlsanity.com/paper/1506.07503/) (2015)
- [MASS: Masked Sequence to Sequence Pre-training for Language Generation](https://mlsanity.com/paper/1905.02450/) (2019)
- [UniLMv2: Pseudo-Masked Language Models for Unified Language Model Pre-Training](https://mlsanity.com/paper/2002.12804/) (2020)
- [RL4F: Generating Natural Language Feedback with Reinforcement Learning for Repairing Model Outputs](https://mlsanity.com/paper/2305.08844/) (2023)
- [Evolved Policy Gradients](https://mlsanity.com/paper/1802.04821/) (2018)
- [A Study of Reinforcement Learning for Neural Machine Translation](https://mlsanity.com/paper/1808.08866/) (2018)
- [Adversarial Generation of Natural Language](https://mlsanity.com/paper/1705.10929/) (2017)
- [Exploring Transformers in Natural Language Generation: GPT, BERT, and XLNet](https://mlsanity.com/paper/2102.08036/) (2021)
- [Implementing the Deep Q-Network](https://mlsanity.com/paper/1711.07478/) (2017)
- [Inverse Reinforcement Learning for Text Summarization](https://mlsanity.com/paper/2212.09917/) (2022)
- [THINK: A Novel Conversation Model for Generating Grammatically Correct and Coherent Responses](https://mlsanity.com/paper/2105.13630/) (2021)
- [Learning Natural Language Generation from Scratch](https://mlsanity.com/paper/2109.09371/) (2021)
- [Adaptive Natural Language Generation for Task-oriented Dialogue via Reinforcement Learning](https://mlsanity.com/paper/2209.07873/) (2022)
- [The Algorithmic Inflection of Russian and Generation of Grammatically Correct Text](https://mlsanity.com/paper/1706.02551/) (2017)

## Abstract (arXiv)

Our research introduces an innovative Natural Language Generation (NLG) approach that aims to optimize user experience and alleviate the workload of human customer support agents. Our primary objective is to generate informal summaries for online articles and posts using an offline reinforcement learning technique. In our study, we compare our proposed method with existing approaches to text generation and provide a comprehensive overview of our architectural design, which incorporates crawling, reinforcement learning, and text generation modules. By presenting this original approach, our paper makes a valuable contribution to the field of NLG by offering a fresh perspective on generating natural language summaries for online content. Through the implementation of Empowering NLG, we are able to generate higher-quality replies in the online domain. The experimental results demonstrate a significant improvement in the average "like" score, increasing from 0.09954378 to 0.5000152. This advancement has the potential to enhance the efficiency and effectiveness of customer support services and elevate the overall user experience when consuming online content.