mlsanityData to 14 Oct 2026

Principled Gradient-based Markov Chain Monte Carlo for Text Generation

arXiv 2312.17710 ↗First posted 29 Dec 2023cs.CL cs.LGcited by 4

Not on the map. Why?

Abstract (arXiv)

Recent papers have demonstrated the possibility of energy-based text generation by adapting gradient-based sampling algorithms, a paradigm of MCMC algorithms that promises fast convergence. However, as we show in this paper, previous attempts on this approach to text generation all fail to sample correctly from the target language model distributions. To address this limitation, we consider the problem of designing text samplers that are faithful, meaning that they have the target text distribution as its limiting distribution. We propose several faithful gradient-based sampling algorithms to sample from the target energy-based text distribution correctly, and study their theoretical properties. Through experiments on various forms of text generation, we demonstrate that faithful samplers are able to generate more fluent text while adhering to the control objectives better.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 23 of its 62 references that are in the map. First posted on arXiv 29 Dec 2023.

Builds on

23 of 62 references.

References · 23

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  3. 2020Improved Techniques for Training Score-Based Generative Modelscited by 1,043
  4. 2019Plug and Play Language Models: A Simple Approach to Controlled Text Generationcited by 799
  5. 2019CTRL: A Conditional Transformer Language Model for Controllable Generationcited by 762
  6. 2020GeDi: Generative Discriminator Guided Sequence Generationcited by 378
  7. 2021FUDGE: Controlled Text Generation With Future Discriminatorscited by 376
  8. 2017The E2E Dataset: New Challenges For End-to-End Generationcited by 253
  9. 2019Composable Effects for Flexible and Accelerated Probabilistic Programming in NumPyrocited by 191
  10. 2022COLD Decoding: Energy-based Constrained Text Generation with Langevin Dynamicscited by 161
  11. 2018Learning to Write with Cooperative Discriminatorscited by 119
  12. 2020Residual Energy-Based Models for Text Generationcited by 95
11 more
  1. 2021Oops I Took A Gradient: Scalable Sampling for Discrete Distributionscited by 92
  2. 2022Mix and Match: Learning-free Controllable Text Generation using Energy Language Modelscited by 66
  3. 2020Limitations of Autoregressive Models and Their Alternativescited by 55
  4. 2022A Langevin-like Sampler for Discrete Distributionscited by 44
  5. 2021Exposing the Implicit Energy Networks behind Masked Language Models via Metropolis--Hastingscited by 37
  6. 2022A Measure-Theoretic Characterization of Tight Language Modelscited by 33
  7. 2016Scan Order in Gibbs Sampling: Models in Which it Matters and Bounds on How Muchcited by 12
  8. 2014Hamiltonian Monte Carlo Without Detailed Balancecited by 10
  9. 2022Enhanced gradient-based MCMC in discrete spacescited by 10
  10. 2013Efficient Random-Walk Methods for Approximating Polytope Volumecited by 3
  11. 2020Geometric algorithms for sampling the flux space of metabolic networkscited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 2; 2025: 0; 2026: 2.

4 papers.

  1. 2024QUEST: Quality-Aware Metropolis-Hastings Sampling for Machine Translationcited by 16
  2. 2024Model-diff: A Tool for Comparative Study of Language Models in the Input Spacecited by 0
  3. 2026Structured Inference with Large Language Gibbscited by 0
  4. 2026BayesPO: Bayesian Prompt Optimization via Parallel-Tempered Gradient-Guided Discrete MCMCcited by 0