mlsanityData to 14 Oct 2026

A Langevin-like Sampler for Discrete Distributions

arXiv 2206.09914 ↗First posted 20 Jun 2022cs.LG stat.MLcited by 44

Not on the map. Why?

Abstract (arXiv)

We propose discrete Langevin proposal (DLP), a simple and scalable gradient-based proposal for sampling complex high-dimensional discrete distributions. In contrast to Gibbs sampling-based methods, DLP is able to update all coordinates in parallel in a single step and the magnitude of changes is controlled by a stepsize. This allows a cheap and efficient exploration in the space of high-dimensional and strongly correlated variables. We prove the efficiency of DLP by showing that the asymptotic bias of its stationary distribution is zero for log-quadratic distributions, and is small for distributions that are close to being log-quadratic. With DLP, we develop several variants of sampling algorithms, including unadjusted, Metropolis-adjusted, stochastic and preconditioned versions. DLP outperforms many popular alternatives on a wide variety of tasks, including Ising models, restricted Boltzmann machines, deep energy-based models, binary neural networks and language generation.

Key facts. Cited by 44 papers in the map (data through 14 Oct 2026). Builds on 26 of its 62 references that are in the map. First posted on arXiv 20 Jun 2022.

Builds on

26 of 62 references.

References · 26

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  3. 2019Generative Modeling by Estimating Gradients of the Data Distributioncited by 4,101
  4. 2016Pointer Sentinel Mixture Modelscited by 3,096
  5. 2015Aligning Books and Movies: Towards Story-like Visual Explanations by Watching Movies and Reading Bookscited by 1,043
  6. 2016XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networkscited by 849
  7. 2018Texygen: A Benchmarking Platform for Text Generation Modelscited by 491
  8. 2017Non-Autoregressive Neural Machine Translationcited by 463
  9. 2019Your Classifier is Secretly an Energy Based Model and You Should Treat it Like Onecited by 405
  10. 2016SeqGAN: Sequence Generative Adversarial Nets with Policy Gradientcited by 376
  11. 2015Probabilistic Backpropagation for Scalable Learning of Bayesian Neural Networkscited by 295
  12. 2017VAE with a VampPriorcited by 248
14 more
  1. 2019Cyclical Stochastic Gradient MCMC for Bayesian Deep Learningcited by 203
  2. 2019BERT has a Mouth, and It Must Speak: BERT as a Markov Random Field Language Modelcited by 185
  3. 2016High-dimensional Bayesian inference via the Unadjusted Langevin Algorithmcited by 146
  4. 2019Understanding Knowledge Distillation in Non-autoregressive Machine Translationcited by 122
  5. 2020Enabling Language Models to Fill in the Blankscited by 122
  6. 2015Preconditioned Stochastic Gradient Langevin Dynamics for Deep Neural Networkscited by 121
  7. 2022Generative Flow Networks for Discrete Probabilistic Modelingcited by 114
  8. 2021Oops I Took A Gradient: Scalable Sampling for Discrete Distributionscited by 92
  9. 2020Post-training Quantization with Multiple Points: Mixed Precision without Mixed Precisioncited by 22
  10. 2018Probabilistic Binary Neural Networkscited by 21
  11. 2015The Hamming Ball Samplercited by 16
  12. 2020Stein Variational Inference for Discrete Distributionscited by 16
  13. 2021Sampling in Combinatorial Spaces with SurVAE Flow Augmented MCMCcited by 7
  14. 2021LSB: Local Self-Balancing MCMC in Discrete Spacescited by 7

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 3; 2023: 9; 2024: 8; 2025: 11; 2026: 13.

44 papers.

Sort:

By area:

  1. 2023Automatically Auditing Large Language Models via Discrete Optimizationcited by 191
  2. 2024Unlocking Guidance for Discrete State-Space Diffusion and Flow Modelscited by 129
  3. 2024Informed Correctors for Discrete Diffusion Modelscited by 57
  4. 2025CANDI: Hybrid Discrete-Continuous Diffusion Modelscited by 34
  5. 2023BayesDAG: Gradient-Based Posterior Inference for Causal Discoverycited by 33
  6. 2022Discrete Langevin Sampler via Wasserstein Gradient Flowcited by 24
  7. 2022Plug & Play Directed Evolution of Proteins with Gradient-based Discrete MCMCcited by 17
  8. 2025Discrete Neural Flow Samplers with Locally Equivariant Transformercited by 13
  9. 2022Enhanced gradient-based MCMC in discrete spacescited by 10
  10. 2025Regularized Langevin Dynamics for Combinatorial Optimizationcited by 10
  11. 2023Masked Bayesian Neural Networks : Theoretical Guarantee and its Posterior Inferencecited by 9
  12. 2024Gradient-based Discrete Sampling with Automatic Cyclical Schedulingcited by 9
32 more
  1. 2023Efficient Informed Proposals for Discrete Distributions via Newton's Series Approximationcited by 7
  2. 2024Optimization by Parallel Quasi-Quantum Annealing with Gradient-Based Samplingcited by 7
  3. 2025Reheated Gradient-based Discrete Sampling for Combinatorial Optimizationcited by 6
  4. 2024Energy-Based Modelling for Discrete and Mixed Data via Heat Equations on Structured Spacescited by 5
  5. 2026Contrastive Distribution Matching for Amortized Sequential Monte Carlo in Discrete Diffusioncited by 5
  6. 2023Non-Log-Concave and Nonsmooth Sampling via Langevin Monte Carlo Algorithmscited by 4
  7. 2023Structured Voronoi Samplingcited by 4
  8. 2023Principled Gradient-based Markov Chain Monte Carlo for Text Generationcited by 4
  9. 2025Gradient GA: Gradient Genetic Algorithm for Drug Molecular Designcited by 4
  10. 2025Differentiable Constraint-Based Causal Discoverycited by 4
  11. 2024Dimension-free Relaxation Times of Informed MCMC Samplers on Discrete Spacescited by 3
  12. 2024Interpolation and differentiation of alchemical degrees of freedom in machine learning interatomic potentialscited by 2
  13. 2024A Family of Distributions of Random Subsets for Controlling Positive and Negative Dependencecited by 2
  14. 2025Entropy-Guided Sampling of Flat Modes in Discrete Spacescited by 2
  15. 2025Learning with Local Search MCMC Layerscited by 2
  16. 2026Discrete diffusion samplers and bridges: Off-policy algorithms and applications in latent spacescited by 2
  17. 2026A Blueprint for Equilibrium-Based Differentiable Continuous-Variable Thermodynamic Computingcited by 2
  18. 2023Evaluation of human-model prediction difference on the Internet Scale of Datacited by 1
  19. 2025Enhancing Gradient-based Discrete Sampling via Parallel Temperingcited by 1
  20. 2025Beyond Self-Repellent Kernels: History-Driven Target Towards Efficient Nonlinear MCMC on General Graphscited by 1
  21. 2026Graph Energy Matching: Transport-Aligned Energy-Based Modeling for Graph Generationcited by 1
  22. 2026Slithering Through Gaps: Capturing Discrete Isolated Modes via Logistic Bridgingcited by 1
  23. 2026Fine-Tuning Low-Bit Models with Gradient in Quantized Code Spacecited by 1
  24. 2023Training Discrete Energy-Based Models with Energy Discrepancycited by 0
  25. 2025Preconditioned Discrete-HAMS: A Second-order Irreversible Discrete Samplercited by 0
  26. 2026Unsupervised Ensemble Learning Through Deep Energy-based Modelscited by 0
  27. 2026Discrete Langevin-Inspired Posterior Samplingcited by 0
  28. 2026Solving Integer Linear Programming with Parallel Temperingcited by 0
  29. 2026Gradient-Guided Reward Optimization for Inference-time Alignmentcited by 0
  30. 2026Iterative Exact Discrete Guidance for Energy-Based Samplingcited by 0
  31. 2026Learning Graph Energies for Finite-Time Generationcited by 0
  32. 2026In-Context Batch Black-box Optimization via Flow Matchingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Scalable Discrete Diffusion Samplers: Combinatorial Optimization and Statistical Physics
  2. 2026Efficient Sampling with Discrete Diffusion Models: Sharp and Adaptive Guarantees
  3. 2024On the Hardness of Sampling from Mixture Distributions via Langevin Dynamics
  4. 2025Unleashing High-Quality Image Generation in Diffusion Sampling Using Second-Order Levenberg-Marquardt-Langevin
  5. 2026Training-Free Generative Sampling via Moment-Matched Score Smoothing
  6. 2026Discrete Feynman-Kac Correctors
  7. 2026Diffusion Removes Langevin's Conditioning Dependence: A Sharp Gaussian Analysis
  8. 2024The Poisson Midpoint Method for Langevin Dynamics: Provably Efficient Discretization for Diffusion Models