mlsanityData to 7 Oct 2026

Training language GANs from Scratch

arXiv 1905.09922 ↗First posted 23 May 2019cs.CL cs.LG stat.MLcited by 31

Not on the map. Why?

Abstract (arXiv)

Generative Adversarial Networks (GANs) enjoy great success at image generation, but have proven difficult to train in the domain of natural language. Challenges with gradient estimation, optimization instability, and mode collapse have lead practitioners to resort to maximum likelihood pre-training, followed by small amounts of adversarial fine-tuning. The benefits of GAN fine-tuning for language generation are unclear, as the resulting models produce comparable or worse samples than traditional language models. We show it is in fact possible to train a language GAN from scratch -- without maximum likelihood pre-training. We combine existing techniques such as large batch sizes, dense rewards and discriminator regularization to stabilize and improve language GANs. The resulting model, ScratchGAN, performs comparably to maximum likelihood training on EMNLP2017 News and WikiText-103 corpora according to quality and diversity metrics.

Key facts. Cited by 31 papers in the map (data through 7 Oct 2026). First posted on arXiv 23 May 2019.

Builds on

No reference list yet.

Built on by

Cited by year

Citing papers per year: 2021: 6; 2022: 10; 2023: 8; 2024: 5; 2025: 2.

31 papers.

Sort:

By area:

  1. 2022Red Teaming Language Models with Language Modelscited by 983
  2. 2023RADAR: Robust AI-Text Detection via Adversarial Learningcited by 187
  3. 2022Self-conditioned Embedding Diffusion for Text Generationcited by 114
  4. 2021Step-unrolled Denoising Autoencoders for Text Generationcited by 102
  5. 2025Target Concrete Score Matching: A Holistic Framework for Discrete Diffusioncited by 25
  6. 2024Countering Reward Over-optimization in LLM with Demonstration-Guided Reinforcement Learningcited by 20
  7. 2022Data Redaction from Pre-trained GANscited by 18
  8. 2021To Beam Or Not To Beam: That is a Question of Cooperation for Language GANscited by 14
  9. 2022Images as Weight Matrices: Sequential Image Generation Through Synaptic Learning Rulescited by 9
  10. 2022Generative Cooperative Networks for Natural Language Generationcited by 8
  11. 2023Stochastic Parrots Looking for Stochastic Parrots: LLMs are Easy to Fine-Tune and Hard to Detect with other LLMscited by 8
  12. 2025Flexible Language Modeling in Continuous Space with Transformer-based Autoregressive Flowscited by 8
19 more
  1. 2022Efficient Reinforcement Learning for Unsupervised Controlled Text Generationcited by 7
  2. 2023Improved Training of Mixture-of-Experts Language GANscited by 7
  3. 2021Regularizing Transformers With Deep Probabilistic Layerscited by 4
  4. 2023In-Context Ability Transfer for Question Decomposition in Complex QAcited by 3
  5. 2021Can the Transformer Be Used as a Drop-in Replacement for RNNs in Text-Generating GANs?cited by 2
  6. 2023Adversarial learning of neural user simulators for dialogue policy optimisationcited by 2
  7. 2023Controlling Federated Learning for Covertnesscited by 2
  8. 2022Why GANs are overkill for NLPcited by 1
  9. 2022Customs Import Declaration Datasetscited by 1
  10. 2022InitialGAN: A Language GAN with Completely Random Initializationcited by 1
  11. 2024Molecular Generative Adversarial Network with Multi-Property Optimizationcited by 1
  12. 2024GANetic Loss for Generative Adversarial Networks with a Focus on Medical Applicationscited by 1
  13. 2024Unstructured Text Enhanced Open-domain Dialogue System: A Systematic Surveycited by 1
  14. 2021Joint Text and Label Generation for Spoken Language Understandingcited by 0
  15. 2021OptAGAN: Entropy-based finetuning on text VAE-GANcited by 0
  16. 2022Learning from Bootstrapping and Stepwise Reinforcement Reward: A Semi-Supervised Framework for Text Style Transfercited by 0
  17. 2023Unlocking the Power of GANs in Non-Autoregressive Text Generationcited by 0
  18. 2023Unsupervised Text Embedding Space Generation Using Generative Adversarial Networks for Text Synthesiscited by 0
  19. 2024Hybrid Generative AI for De Novo Design of Co-Crystals with Enhanced Tabletabilitycited by 0