# GANS for Sequences of Discrete Elements with the Gumbel-softmax Distribution

arXiv: https://arxiv.org/abs/1611.04051 · HTML: https://mlsanity.com/paper/1611.04051/

## Key facts

- Cited by 48 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Nov 2016.

## Built on by (most cited first, 30 of 48)

- [A Review of the Gumbel-max Trick and its Extensions for Discrete Stochasticity in Machine Learning](https://mlsanity.com/paper/2110.01515/) (2021)
- [Discrete-state Continuous-time Diffusion for Graph Generation](https://mlsanity.com/paper/2405.11416/) (2024)
- [Eco2AI: carbon emissions tracking of machine learning models as the first step towards sustainable AI](https://mlsanity.com/paper/2208.00406/) (2022)
- [Math Word Problem Generation with Mathematical Consistency and Problem Context Constraints](https://mlsanity.com/paper/2109.04546/) (2021)
- [Musika! Fast Infinite Waveform Music Generation](https://mlsanity.com/paper/2208.08706/) (2022)
- [A survey on text generation using generative adversarial networks](https://mlsanity.com/paper/2212.11119/) (2022)
- [Data Imputation with Iterative Graph Reconstruction](https://mlsanity.com/paper/2212.02810/) (2022)
- [Semantic Palette: Guiding Scene Generation with Class Proportions](https://mlsanity.com/paper/2106.01629/) (2021)
- [A review of Generative Adversarial Networks (GANs) and its applications in a wide variety of disciplines -- From Medical to Remote Sensing](https://mlsanity.com/paper/2110.01442/) (2021)
- [Efficient Generator of Mathematical Expressions for Symbolic Regression](https://mlsanity.com/paper/2302.09893/) (2023)
- [Generating music with sentiment using Transformer-GANs](https://mlsanity.com/paper/2212.11134/) (2022)
- [Unified Generation, Reconstruction, and Representation: Generalized Diffusion with Adaptive Latent Encoding-Decoding](https://mlsanity.com/paper/2402.19009/) (2024)
- [SUPR-GAN: SUrgical PRediction GAN for Event Anticipation in Laparoscopic and Robotic Surgery](https://mlsanity.com/paper/2105.04642/) (2021)
- [Improved Training of Mixture-of-Experts Language GANs](https://mlsanity.com/paper/2302.11875/) (2023)
- [V2Flow: Unifying Visual Tokenization and Large Language Model Vocabularies for Autoregressive Image Generation](https://mlsanity.com/paper/2503.07493/) (2025)
- [Soft-Di[M]O: Improving One-Step Discrete Image Generation with Soft Embeddings](https://mlsanity.com/paper/2509.22925/) (2025)
- [Generating Symbolic Reasoning Problems with Transformer GANs](https://mlsanity.com/paper/2110.10054/) (2021)
- [The Power of Fragmentation: A Hierarchical Transformer Model for Structural Segmentation in Symbolic Music Generation](https://mlsanity.com/paper/2205.08579/) (2022)
- [Predicting Opinion Dynamics via Sociologically-Informed Neural Networks](https://mlsanity.com/paper/2207.03990/) (2022)
- [Categorical EHR Imputation with Generative Adversarial Nets](https://mlsanity.com/paper/2108.01701/) (2021)
- [Revisiting the Gumbel-Softmax in MADDPG](https://mlsanity.com/paper/2302.11793/) (2023)
- [Adversarial Machine Learning in Text Analysis and Generation](https://mlsanity.com/paper/2101.08675/) (2021)
- [Emojich -- zero-shot emoji generation using Russian language: a technical report](https://mlsanity.com/paper/2112.02448/) (2021)
- [Improvement in Machine Translation with Generative Adversarial Networks](https://mlsanity.com/paper/2111.15166/) (2021)
- [Survey of Generative Methods for Social Media Analysis](https://mlsanity.com/paper/2112.07041/) (2021)
- [FinGAN: Generative Adversarial Network for Analytical Customer Relationship Management in Banking and Insurance](https://mlsanity.com/paper/2201.11486/) (2022)
- [Why GANs are overkill for NLP](https://mlsanity.com/paper/2205.09838/) (2022)
- [AdaTriplet-RA: Domain Matching via Adaptive Triplet and Reinforced Attention for Unsupervised Domain Adaptation](https://mlsanity.com/paper/2211.08894/) (2022)
- [A Mutation-based Text Generation for Adversarial Machine Learning Applications](https://mlsanity.com/paper/2212.11808/) (2022)
- [Modified Query Expansion Through Generative Adversarial Networks for Information Extraction in E-Commerce](https://mlsanity.com/paper/2301.00036/) (2022)

## Abstract (arXiv)

Generative Adversarial Networks (GAN) have limitations when the goal is to generate sequences of discrete elements. The reason for this is that samples from a distribution on discrete objects such as the multinomial are not differentiable with respect to the distribution parameters. This problem can be avoided by using the Gumbel-softmax distribution, which is a continuous approximation to a multinomial distribution parameterized in terms of the softmax function. In this work, we evaluate the performance of GANs based on recurrent neural networks with Gumbel-softmax output distributions in the task of generating sequences of discrete elements.