# Language Generation with Recurrent Generative Adversarial Networks without Pre-training

arXiv: https://arxiv.org/abs/1706.01399 · HTML: https://mlsanity.com/paper/1706.01399/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 5 Jun 2017.

## Built on by (most cited first, 14 of 14)

- [Continuous 3D Multi-Channel Sign Language Production via Progressive Transformers and Mixture Density Networks](https://mlsanity.com/paper/2103.06982/) (2021)
- [The Health Gym: Synthetic Health-Related Datasets for the Development of Reinforcement Learning Algorithms](https://mlsanity.com/paper/2203.06369/) (2022)
- [A survey on text generation using generative adversarial networks](https://mlsanity.com/paper/2212.11119/) (2022)
- [On the Effectiveness of Pretrained Models for API Learning](https://mlsanity.com/paper/2204.03498/) (2022)
- [Continual Learning with Dirichlet Generative-based Rehearsal](https://mlsanity.com/paper/2309.06917/) (2023)
- [SPI-GAN: Towards Single-Pixel Imaging through Generative Adversarial Network](https://mlsanity.com/paper/2107.01330/) (2021)
- [Adversarial Transformer Language Models for Contextual Commonsense Inference](https://mlsanity.com/paper/2302.05406/) (2023)
- [Why GANs are overkill for NLP](https://mlsanity.com/paper/2205.09838/) (2022)
- [ObjBlur: A Curriculum Learning Approach With Progressive Object-Level Blurring for Improved Layout-to-Image Generation](https://mlsanity.com/paper/2404.07564/) (2024)
- [Taming the Beast: Learning to Control Neural Conversational Models](https://mlsanity.com/paper/2108.10561/) (2021)
- [OptAGAN: Entropy-based finetuning on text VAE-GAN](https://mlsanity.com/paper/2109.00239/) (2021)
- [Improving Model Compatibility of Generative Adversarial Networks by Boundary Calibration](https://mlsanity.com/paper/2111.02316/) (2021)
- [Noun2Verb: Probabilistic frame semantics for word class conversion](https://mlsanity.com/paper/2205.06321/) (2022)
- [Unsupervised Text Embedding Space Generation Using Generative Adversarial Networks for Text Synthesis](https://mlsanity.com/paper/2306.17181/) (2023)

## Abstract (arXiv)

Generative Adversarial Networks (GANs) have shown great promise recently in image generation. Training GANs for language generation has proven to be more difficult, because of the non-differentiable nature of generating text with recurrent neural networks. Consequently, past work has either resorted to pre-training with maximum-likelihood or used convolutional networks for generation. In this work, we show that recurrent neural networks can be trained to generate text with GANs from scratch using curriculum learning, by slowly teaching the model to generate sequences of increasing and variable length. We empirically show that our approach vastly improves the quality of generated sequences compared to a convolutional baseline.