# ARAML: A Stable Adversarial Training Framework for Text Generation

arXiv: https://arxiv.org/abs/1908.07195 · HTML: https://mlsanity.com/paper/1908.07195/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Aug 2019.

## Built on by (most cited first, 9 of 9)

- [Inverse-RLignment: Large Language Model Alignment from Demonstrations through Inverse Reinforcement Learning](https://mlsanity.com/paper/2405.15624/) (2024)
- [Imitating Language via Scalable Inverse Reinforcement Learning](https://mlsanity.com/paper/2409.01369/) (2024)
- [Reinforcement Learning for Generative AI: A Survey](https://mlsanity.com/paper/2308.14328/) (2023)
- [Improved Training of Mixture-of-Experts Language GANs](https://mlsanity.com/paper/2302.11875/) (2023)
- [Learning to Augment for Data-Scarce Domain BERT Knowledge Distillation](https://mlsanity.com/paper/2101.08106/) (2021)
- [SparseGAN: Sparse Generative Adversarial Network for Text Generation](https://mlsanity.com/paper/2103.11578/) (2021)
- [InitialGAN: A Language GAN with Completely Random Initialization](https://mlsanity.com/paper/2208.02531/) (2022)
- [Pluggable Neural Machine Translation Models via Memory-augmented Adapters](https://mlsanity.com/paper/2307.06029/) (2023)
- [Differential Privacy: What is all the noise about?](https://mlsanity.com/paper/2205.09453/) (2022)

## Abstract (arXiv)

Most of the existing generative adversarial networks (GAN) for text generation suffer from the instability of reinforcement learning training algorithms such as policy gradient, leading to unstable performance. To tackle this problem, we propose a novel framework called Adversarial Reward Augmented Maximum Likelihood (ARAML). During adversarial training, the discriminator assigns rewards to samples which are acquired from a stationary distribution near the data rather than the generator's distribution. The generator is optimized with maximum likelihood estimation augmented by the discriminator's rewards instead of policy gradient. Experiments show that our model can outperform state-of-the-art text GANs with a more stable training process.