# Generative Cooperative Networks for Natural Language Generation

arXiv: https://arxiv.org/abs/2201.12320 · HTML: https://mlsanity.com/paper/2201.12320/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 29 of its 39 references that are in the map.
- First posted on arXiv 28 Jan 2022.

## Builds on (most cited first, 29 of 29)

- [Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer](https://mlsanity.com/paper/1910.10683/) (2019)
- [SQuAD: 100,000+ Questions for Machine Comprehension of Text](https://mlsanity.com/paper/1606.05250/) (2016)
- [Sequence to Sequence Learning with Neural Networks](https://mlsanity.com/paper/1409.3215/) (2014)
- [The Curious Case of Neural Text Degeneration](https://mlsanity.com/paper/1904.09751/) (2019)
- [HuggingFace's Transformers: State-of-the-art Natural Language Processing](https://mlsanity.com/paper/1910.03771/) (2019)
- [Self-Attention with Relative Position Representations](https://mlsanity.com/paper/1803.02155/) (2018)
- [Abstractive Text Summarization Using Sequence-to-Sequence RNNs and Beyond](https://mlsanity.com/paper/1602.06023/) (2016)
- [Scheduled Sampling for Sequence Prediction with Recurrent Neural Networks](https://mlsanity.com/paper/1506.03099/) (2015)
- [Sequence Level Training with Recurrent Neural Networks](https://mlsanity.com/paper/1511.06732/) (2015)
- [Neural Text Generation with Unlikelihood Training](https://mlsanity.com/paper/1908.04319/) (2019)
- [SeqGAN: Sequence Generative Adversarial Nets with Policy Gradient](https://mlsanity.com/paper/1609.05473/) (2016)
- [A Deep Reinforced Model for Abstractive Summarization](https://mlsanity.com/paper/1705.04304/) (2017)
- [Thinking Fast and Slow with Deep Learning and Tree Search](https://mlsanity.com/paper/1705.08439/) (2017)
- [Noise Contrastive Estimation and Negative Sampling for Conditional Models: Consistency and Statistical Efficiency](https://mlsanity.com/paper/1809.01812/) (2018)
- [Long Text Generation via Adversarial Training with Leaked Information](https://mlsanity.com/paper/1709.08624/) (2017)
- [Language GANs Falling Short](https://mlsanity.com/paper/1811.02549/) (2018)
- [Residual Energy-Based Models for Text Generation](https://mlsanity.com/paper/2004.11714/) (2020)
- [Answers Unite! Unsupervised Metrics for Reinforced Summarization Models](https://mlsanity.com/paper/1909.01610/) (2019)
- [Reward Augmented Maximum Likelihood for Neural Structured Prediction](https://mlsanity.com/paper/1609.00150/) (2016)
- [Machine Translation Decoding beyond Beam Search](https://mlsanity.com/paper/2104.05336/) (2021)
- [Maximum-Likelihood Augmented Discrete Generative Adversarial Networks](https://mlsanity.com/paper/1702.07983/) (2017)
- [Training language GANs from Scratch](https://mlsanity.com/paper/1905.09922/) (2019)
- [Boundary-Seeking Generative Adversarial Networks](https://mlsanity.com/paper/1702.08431/) (2017)
- [Discriminative Adversarial Search for Abstractive Summarization](https://mlsanity.com/paper/2002.10375/) (2020)
- [To Beam Or Not To Beam: That is a Question of Cooperation for Language GANs](https://mlsanity.com/paper/2106.06363/) (2021)
- [ColdGANs: Taming Language GANs with Cautious Sampling Strategies](https://mlsanity.com/paper/2006.04643/) (2020)
- [Self-Adversarial Learning with Comparative Discrimination for Text Generation](https://mlsanity.com/paper/2001.11691/) (2020)
- [Learning Beam Search Policies via Imitation Learning](https://mlsanity.com/paper/1811.00512/) (2018)
- [Improving Maximum Likelihood Training for Text Generation with Density Ratio Estimation](https://mlsanity.com/paper/2007.06018/) (2020)

## Built on by (most cited first, 8 of 8)

- [Quark: Controllable Text Generation with Reinforced Unlearning](https://mlsanity.com/paper/2205.13636/) (2022)
- [KCTS: Knowledge-Constrained Tree Search Decoding with Token-Level Hallucination Detection](https://mlsanity.com/paper/2310.09044/) (2023)
- [Reinforcement Learning for Generative AI: A Survey](https://mlsanity.com/paper/2308.14328/) (2023)
- [Reinforcement Learning for Generative AI: State of the Art, Opportunities and Open Research Challenges](https://mlsanity.com/paper/2308.00031/) (2023)
- [Joint Generator-Ranker Learning for Natural Language Generation](https://mlsanity.com/paper/2206.13974/) (2022)
- [Which Discriminator for Cooperative Text Generation?](https://mlsanity.com/paper/2204.11586/) (2022)
- [Enhancing Text Generation with Cooperative Training](https://mlsanity.com/paper/2303.09075/) (2023)
- [Unsupervised Event Outlier Detection in Continuous Time](https://mlsanity.com/paper/2411.16427/) (2024)

## Abstract (arXiv)

Generative Adversarial Networks (GANs) have known a tremendous success for many continuous generation tasks, especially in the field of image generation. However, for discrete outputs such as language, optimizing GANs remains an open problem with many instabilities, as no gradient can be properly back-propagated from the discriminator output to the generator parameters. An alternative is to learn the generator network via reinforcement learning, using the discriminator signal as a reward, but such a technique suffers from moving rewards and vanishing gradient problems. Finally, it often falls short compared to direct maximum-likelihood approaches. In this paper, we introduce Generative Cooperative Networks, in which the discriminator architecture is cooperatively used along with the generation policy to output samples of realistic texts for the task at hand. We give theoretical guarantees of convergence for our approach, and study various efficient decoding schemes to empirically achieve state-of-the-art results in two main NLG tasks.