mlsanityData to 14 Oct 2026

A Review of the Gumbel-max Trick and its Extensions for Discrete Stochasticity in Machine Learning

arXiv 2110.01515 ↗First posted 4 Oct 2021cs.LG stat.MLcited by 82

Not on the map. Why?

Abstract (arXiv)

The Gumbel-max trick is a method to draw a sample from a categorical distribution, given by its unnormalized (log-)probabilities. Over the past years, the machine learning community has proposed several extensions of this trick to facilitate, e.g., drawing multiple samples, sampling from structured domains, or gradient estimation for error backpropagation in neural network optimization. The goal of this survey article is to present background about the Gumbel-max trick, and to provide a structured overview of its extensions to ease algorithm selection. Moreover, it presents a comprehensive outline of (machine learning) literature in which Gumbel-based algorithms have been leveraged, reviews commonly-made design choices, and sketches a future perspective.

Key facts. Cited by 82 papers in the map (data through 14 Oct 2026). Builds on 93 of its 172 references that are in the map. First posted on arXiv 4 Oct 2021.

Builds on

93 of 172 references.

References · 93

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  2. 2017Neural Discrete Representation Learningcited by 4,712
  3. 2021Zero-Shot Text-to-Image Generationcited by 4,288
  4. 2019The Curious Case of Neural Text Degenerationcited by 2,916
  5. 2016Categorical Reparameterization with Gumbel-Softmaxcited by 2,739
  6. 2013Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computationcited by 2,079
  7. 2018Hierarchical Neural Story Generationcited by 1,193
  8. 2016The Concrete Distribution: A Continuous Relaxation of Discrete Random Variablescited by 1,148
  9. 2016Pixel Recurrent Neural Networkscited by 816
  10. 2017OptNet: Differentiable Optimization as a Layer in Neural Networkscited by 599
  11. 2017Learning Sparse Neural Networks through L0 Regularizationcited by 583
  12. 2018FBNet: Hardware-Aware Efficient ConvNet Design via Differentiable Neural Architecture Searchcited by 485
38 more
  1. 2019vq-wav2vec: Self-Supervised Learning of Discrete Speech Representationscited by 340
  2. 2015Improved Semantic Representations From Tree-Structured Long Short-Term Memory Networkscited by 306
  3. 2018SNAS: Stochastic Neural Architecture Searchcited by 289
  4. 2019Monte Carlo Gradient Estimation in Machine Learningcited by 283
  5. 2017Emergence of Grounded Compositional Language in Multi-Agent Populationscited by 263
  6. 2018Learning to Explain: An Information-Theoretic Perspective on Model Interpretationcited by 248
  7. 2017Concrete Dropoutcited by 227
  8. 2019Stochastic Beams and Where to Find Them: The Gumbel-Top-k Trick for Sampling Sequences Without Replacementcited by 211
  9. 2018Learning Latent Permutations with Gumbel-Sinkhorn Networkscited by 175
  10. 2020Interpreting Graph Neural Networks for NLP With Differentiable Edge Maskingcited by 153
  11. 2020FBNetV2: Differentiable Neural Architecture Search for Spatial and Channel Dimensionscited by 152
  12. 2014A* Samplingcited by 151
  13. 2019Modeling Point Clouds with Self-Attention and Gumbel Subset Samplingcited by 151
  14. 2017Convolutional Networks with Adaptive Inference Graphscited by 148
  15. 2018Learning to Assemble Neural Module Tree Networks for Visual Groundingcited by 140
  16. 2017Emergence of Language with Multi-agent Games: Learning to Communicate with Sequences of Symbolscited by 125
  17. 2014Neural Variational Inference and Learning in Belief Networkscited by 117
  18. 2019Counterfactual Off-Policy Evaluation with Gumbel-Max Structural Causal Modelscited by 117
  19. 2020Learning to Branch for Multi-Task Learningcited by 115
  20. 2019Stochastic Optimization of Sorting Networks via Continuous Relaxationscited by 111
  21. 2017Backpropagation through the Void: Optimizing control variates for black-box gradient estimationcited by 106
  22. 2017Towards a Visual Privacy Advisor: Understanding and Predicting Privacy Risks in Imagescited by 101
  23. 2019Dynamic Convolutions: Exploiting Spatial Sparsity for Faster Inferencecited by 91
  24. 2019Concrete Autoencoders for Differentiable Feature Selection and Reconstructioncited by 90
  25. 2016Variational inference for Monte Carlo objectivescited by 84
  26. 2018Learning Disentangled Joint Continuous and Discrete Representationscited by 83
  27. 2020Improving Inference for Neural Image Compressioncited by 83
  28. 2020Learning with Differentiable Perturbed Optimizerscited by 81
  29. 2019Network Pruning via Transformable Architecture Searchcited by 80
  30. 2017REBAR: Low-variance, unbiased gradient estimates for discrete latent variable modelscited by 79
  31. 2019Deep Declarative Networks: A New Hopecited by 78
  32. 2020Operation-Aware Soft Channel Pruning using Differentiable Maskscited by 74
  33. 2019Reparameterizable Subset Sampling via Continuous Relaxationscited by 70
  34. 2017Boltzmann Exploration Done Rightcited by 69
  35. 2020Gradient Estimation with Stochastic Softmax Trickscited by 67
  36. 2018SparseMAP: Differentiable Sparse Structured Inferencecited by 62
  37. 2017Classical Planning in Deep Latent Space: Bridging the Subsymbolic-Symbolic Boundarycited by 58
  38. 2016GANS for Sequences of Discrete Elements with the Gumbel-softmax Distributioncited by 48

50 most cited of 93 All 93 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 1; 2022: 9; 2023: 18; 2024: 18; 2025: 28; 2026: 8.

82 papers.

Sort:

By area:

  1. 2024Self-Improvement for Neural Combinatorial Optimization: Sample without Replacement, but Improvementcited by 34
  2. 2025Efficient-DLM: From Autoregressive to Diffusion Language Models, and Beyond in Speedcited by 33
  3. 2025Advancing Sequential Numerical Prediction in Autoregressive Modelscited by 26
  4. 2024Residual Quantization with Implicit Neural Codebookscited by 25
  5. 2022Sparse Graph Learning from Spatiotemporal Time Seriescited by 22
  6. 2022Differentiable Agent-based Epidemiologycited by 21
  7. 2021Counterfactual Temporal Point Processescited by 20
  8. 2023A skeletonization algorithm for gradient-based optimizationcited by 17
  9. 2023Differentiable Clustering with Perturbed Spanning Forestscited by 14
  10. 2023Differentiable Sampling of Categorical Distributions Using the CatLog-Derivative Trickcited by 14
  11. 2024MEGA: Masked Generative Autoencoder for Human Mesh Recoverycited by 14
  12. 2022Explain My Surprise: Learning Efficient Long-Term Memory by Predicting Uncertain Outcomescited by 11
38 more
  1. 2024Conditional computation in neural networks: principles and research trendscited by 10
  2. 2023Prototype-based Dataset Comparisoncited by 9
  3. 2025Evaluation of Large Language Models via Coupled Token Generationcited by 8
  4. 2023GRAPES: Learning to Sample Graphs for Scalable Graph Neural Networkscited by 7
  5. 2024Bypass Back-propagation: Optimization-based Structural Pruning for Large Language Models via Policy Gradientcited by 6
  6. 2024Coupling without Communication and Drafter-Invariant Speculative Decodingcited by 6
  7. 2024A Review of Bayesian Uncertainty Quantification in Deep Probabilistic Image Segmentationcited by 6
  8. 2022Learning Group Importance using the Differentiable Hypergeometric Distributioncited by 5
  9. 2022Predictive Querying for Autoregressive Neural Sequence Modelscited by 5
  10. 2023Improving Dual-Encoder Training through Dynamic Indexes for Negative Miningcited by 5
  11. 2024Understanding What Affects the Generalization Gap in Visual Reinforcement Learning: Theory and Empirical Evidencecited by 5
  12. 2025NLGR: Utilizing Neighbor Lists for Generative Rerank in Personalized Recommendation Systemscited by 5
  13. 2025SketchDNN: Joint Continuous-Discrete Diffusion for CAD Sketch Generationcited by 5
  14. 2022Fast Offline Policy Optimization for Large Scale Recommendationcited by 4
  15. 2025FROG: Fair Removal on Graphscited by 4
  16. 2025A Simple Contrastive Framework Of Item Tokenization For Generative Recommendationcited by 4
  17. 2025Neural Image Abstraction Using Long Smoothing B-Splinescited by 4
  18. 2022Causal Temporal Reasoning for Markov Decision Processescited by 3
  19. 2023Revisiting the Gumbel-Softmax in MADDPGcited by 3
  20. 2023Single Node Injection Label Specificity Attack on Graph Neural Networks via Reinforcement Learningcited by 3
  21. 2023Environmental effects on emergent strategy in micro-scale multi-agent reinforcement learningcited by 3
  22. 2023Fast Slate Policy Optimization: Going Beyond Plackett-Lucecited by 3
  23. 2024A Literature Review of Literature Reviews in Pattern Analysis and Machine Intelligencecited by 3
  24. 2025VideoCompressa: Data-Efficient Video Understanding via Joint Temporal Compression and Spatial Reconstructioncited by 3
  25. 2025Masked Generative Policy for Robotic Controlcited by 3
  26. 2026FlashSampling: Fast and Memory-Efficient Exact Samplingcited by 3
  27. 2023Some challenges of calibrating differentiable agent-based modelscited by 2
  28. 2023Mixture-of-Linguistic-Experts Adapters for Improving and Interpreting Pre-trained Language Modelscited by 2
  29. 2024Unifying Graph Contrastive Learning via Graph Message Augmentationcited by 2
  30. 2024Direct Acquisition Optimization for Low-Budget Active Learningcited by 2
  31. 2025Rethinking Approximate Gaussian Inference in Classificationcited by 2
  32. 2025A Multi-Fidelity Control Variate Approach for Policy Gradient Estimationcited by 2
  33. 2025DiFR: Inference Verification Despite Nondeterminismcited by 2
  34. 2023Compressing Sentence Representation with maximum Coding Rate Reductioncited by 1
  35. 2023Triggering Dark Showers with Conditional Dual Auto-Encoderscited by 1
  36. 2023Causal Dynamic Variational Autoencoder for Counterfactual Regression in Longitudinal Datacited by 1
  37. 2024Bayesian Unsupervised Disentanglement of Anatomy and Geometry for Deep Groupwise Image Registrationcited by 1
  38. 2024Neural Click Models for Recommender Systemscited by 1

50 most cited of 82 All 82 (JSON)