# SparseGAN: Sparse Generative Adversarial Network for Text Generation

arXiv: https://arxiv.org/abs/2103.11578 · HTML: https://mlsanity.com/paper/2103.11578/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 45 references that are in the map.
- First posted on arXiv 22 Mar 2021.

## Builds on (most cited first, 17 of 17)

- [Categorical Reparameterization with Gumbel-Softmax](https://mlsanity.com/paper/1611.01144/) (2016)
- [Improved Training of Wasserstein GANs](https://mlsanity.com/paper/1704.00028/) (2017)
- [Microsoft COCO Captions: Data Collection and Evaluation Server](https://mlsanity.com/paper/1504.00325/) (2015)
- [Generating Sentences from a Continuous Space](https://mlsanity.com/paper/1511.06349/) (2015)
- [Sequence Level Training with Recurrent Neural Networks](https://mlsanity.com/paper/1511.06732/) (2015)
- [Texygen: A Benchmarking Platform for Text Generation Models](https://mlsanity.com/paper/1802.01886/) (2018)
- [SeqGAN: Sequence Generative Adversarial Nets with Policy Gradient](https://mlsanity.com/paper/1609.05473/) (2016)
- [Long Text Generation via Adversarial Training with Leaked Information](https://mlsanity.com/paper/1709.08624/) (2017)
- [MaskGAN: Better Text Generation via Filling in the______](https://mlsanity.com/paper/1801.07736/) (2018)
- [Adversarial Feature Matching for Text Generation](https://mlsanity.com/paper/1706.03850/) (2017)
- [Adversarial Ranking for Language Generation](https://mlsanity.com/paper/1705.11001/) (2017)
- [Maximum-Likelihood Augmented Discrete Generative Adversarial Networks](https://mlsanity.com/paper/1702.07983/) (2017)
- [Unsupervised Natural Language Generation with Denoising Autoencoders](https://mlsanity.com/paper/1804.07899/) (2018)
- [ARAML: A Stable Adversarial Training Framework for Text Generation](https://mlsanity.com/paper/1908.07195/) (2019)
- [Latent Code and Text-based Generative Adversarial Networks for Soft-text Generation](https://mlsanity.com/paper/1904.07293/) (2019)
- [Adversarially Regularized Autoencoders](https://mlsanity.com/paper/1706.04223/) (2017)
- [Generative Adversarial Networks for text using word2vec intermediaries](https://mlsanity.com/paper/1904.02293/) (2019)

## Built on by (most cited first, 2 of 2)

- [Robust Natural Language Processing: Recent Advances, Challenges, and Future Directions](https://mlsanity.com/paper/2201.00768/) (2022)
- [Layer-Wise Perturbations via Sparse Autoencoders for Adversarial Text Generation](https://mlsanity.com/paper/2508.10404/) (2025)

## Abstract (arXiv)

It is still a challenging task to learn a neural text generation model under the framework of generative adversarial networks (GANs) since the entire training process is not differentiable. The existing training strategies either suffer from unreliable gradient estimations or imprecise sentence representations. Inspired by the principle of sparse coding, we propose a SparseGAN that generates semantic-interpretable, but sparse sentence representations as inputs to the discriminator. The key idea is that we treat an embedding matrix as an over-complete dictionary, and use a linear combination of very few selected word embeddings to approximate the output feature representation of the generator at each time step. With such semantic-rich representations, we not only reduce unnecessary noises for efficient adversarial training, but also make the entire training process fully differentiable. Experiments on multiple text generation datasets yield performance improvements, especially in sequence-level metrics, such as BLEU.