# TextGAIL: Generative Adversarial Imitation Learning for Text Generation

arXiv: https://arxiv.org/abs/2004.13796 · HTML: https://mlsanity.com/paper/2004.13796/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 7 Apr 2020.

## Built on by (most cited first, 28 of 28)

- [Is Reinforcement Learning (Not) for Natural Language Processing: Benchmarks, Baselines, and Building Blocks for Natural Language Policy Optimization](https://mlsanity.com/paper/2210.01241/) (2022)
- [Automatically Correcting Large Language Models: Surveying the landscape of diverse self-correction strategies](https://mlsanity.com/paper/2308.03188/) (2023)
- [RADAR: Robust AI-Text Detection via Adversarial Learning](https://mlsanity.com/paper/2307.03838/) (2023)
- [StyleGAN-Human: A Data-Centric Odyssey of Human Generation](https://mlsanity.com/paper/2204.11823/) (2022)
- [PLACES: Prompting Language Models for Social Conversation Synthesis](https://mlsanity.com/paper/2302.03269/) (2023)
- [Adversarial Preference Optimization: Enhancing Your Alignment via RM-LLM Game](https://mlsanity.com/paper/2311.08045/) (2023)
- [Inverse-RLignment: Large Language Model Alignment from Demonstrations through Inverse Reinforcement Learning](https://mlsanity.com/paper/2405.15624/) (2024)
- [Generating Summaries with Controllable Readability Levels](https://mlsanity.com/paper/2310.10623/) (2023)
- [Imitating Language via Scalable Inverse Reinforcement Learning](https://mlsanity.com/paper/2409.01369/) (2024)
- [A survey on text generation using generative adversarial networks](https://mlsanity.com/paper/2212.11119/) (2022)
- [Reinforcement Learning for Generative AI: A Survey](https://mlsanity.com/paper/2308.14328/) (2023)
- [KRLS: Improving End-to-End Response Generation in Task Oriented Dialog with Reinforced Keywords Learning](https://mlsanity.com/paper/2211.16773/) (2022)
- [SGIC: A Self-Guided Iterative Calibration Framework for RAG](https://mlsanity.com/paper/2506.16172/) (2025)
- [How Deep Are the Fakes? Focusing on Audio Deepfake: A Survey](https://mlsanity.com/paper/2111.14203/) (2021)
- [Summary Grounded Conversation Generation](https://mlsanity.com/paper/2106.03337/) (2021)
- [Adaptive Bridge between Training and Inference for Dialogue](https://mlsanity.com/paper/2110.11560/) (2021)
- [From Demonstrations to Rewards: Alignment Without Explicit Human Preferences](https://mlsanity.com/paper/2503.13538/) (2025)
- [A Single Example Can Improve Zero-Shot Data Generation](https://mlsanity.com/paper/2108.06991/) (2021)
- [Adversarial learning of neural user simulators for dialogue policy optimisation](https://mlsanity.com/paper/2306.00858/) (2023)
- [An Empirical Study on Self-correcting Large Language Models for Data Science Code Generation](https://mlsanity.com/paper/2408.15658/) (2024)
- [Language Model Distillation: A Temporal Difference Imitation Learning Perspective](https://mlsanity.com/paper/2505.20335/) (2025)
- [On Combining Expert Demonstrations in Imitation Learning via Optimal Transport](https://mlsanity.com/paper/2307.10810/) (2023)
- [Latent Adversarial Regularization for Offline Preference Optimization](https://mlsanity.com/paper/2601.22083/) (2026)
- [Generative Adversarial Imitation Learning for Empathy-based AI](https://mlsanity.com/paper/2105.13328/) (2021)
- [OptAGAN: Entropy-based finetuning on text VAE-GAN](https://mlsanity.com/paper/2109.00239/) (2021)
- [Enhancing Text Generation with Cooperative Training](https://mlsanity.com/paper/2303.09075/) (2023)
- [Extrinsicaly Rewarded Soft Q Imitation Learning with Discriminator](https://mlsanity.com/paper/2401.16772/) (2024)
- [Imitation Learning in the Deep Learning Era: A Novel Taxonomy and Recent Advances](https://mlsanity.com/paper/2511.03565/) (2025)

## Abstract (arXiv)

Generative Adversarial Networks (GANs) for text generation have recently received many criticisms, as they perform worse than their MLE counterparts. We suspect previous text GANs' inferior performance is due to the lack of a reliable guiding signal in their discriminators. To address this problem, we propose a generative adversarial imitation learning framework for text generation that uses large pre-trained language models to provide more reliable reward guidance. Our approach uses contrastive discriminator, and proximal policy optimization (PPO) to stabilize and improve text generation performance. For evaluation, we conduct experiments on a diverse set of unconditional and conditional text generation tasks. Experimental results show that TextGAIL achieves better performance in terms of both quality and diversity than the MLE baseline. We also validate our intuition that TextGAIL's discriminator demonstrates the capability of providing reasonable rewards with an additional task.