mlsanityData to 14 Oct 2026

Semantic Probabilistic Control of Language Models

arXiv 2505.01954 ↗First posted 4 May 2025cs.LGcited by 4

Not on the map. Why?

Abstract (arXiv)

Semantic control entails steering LM generations towards satisfying subtle non-lexical constraints, e.g., toxicity, sentiment, or politeness, attributes that can be captured by a sequence-level verifier. It can thus be viewed as sampling from the LM distribution conditioned on the target attribute, a computationally intractable problem due to the non-decomposable nature of the verifier. Existing approaches to LM control either only deal with syntactic constraints which cannot capture the aforementioned attributes, or rely on sampling to explore the conditional LM distribution, an ineffective estimator for low-probability events. In this work, we leverage a verifier's gradient information to efficiently reason over all generations that satisfy the target attribute, enabling precise steering of LM generations by reweighing the next-token distribution. Starting from an initial sample, we create a local LM distribution favoring semantically similar sentences. This approximation enables the tractable computation of an expected sentence embedding. We use this expected embedding, informed by the verifier's evaluation at the initial sample, to estimate the probability of satisfying the constraint, which directly informs the update to the next-token distribution. We evaluated the effectiveness of our approach in controlling the toxicity, sentiment, and topic-adherence of LMs yielding generations satisfying the constraint with high probability (>95%) without degrading their quality.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 60 of its 77 references that are in the map. First posted on arXiv 4 May 2025.

Builds on

60 of 77 references.

References · 60

  1. 2022Training language models to follow instructions with human feedbackcited by 15,971
  2. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  3. 2019The Curious Case of Neural Text Degenerationcited by 2,916
  4. 2020Learning to summarize from human feedbackcited by 2,588
  5. 2019Fine-Tuning Language Models from Human Preferencescited by 2,150
  6. 2022Fast Inference from Transformers via Speculative Decodingcited by 1,653
  7. 2020RealToxicityPrompts: Evaluating Neural Toxic Degeneration in Language Modelscited by 1,255
  8. 2019Plug and Play Language Models: A Simple Approach to Controlled Text Generationcited by 799
  9. 2019Universal Adversarial Triggers for Attacking and Analyzing NLPcited by 722
  10. 2022Contrastive Decoding: Open-ended Text Generation as Optimizationcited by 609
  11. 2024Smarter, Better, Faster, Longer: A Modern Bidirectional Encoder for Fast, Memory Efficient, and Long Context Finetuning and Inferencecited by 554
  12. 2021DExperts: Decoding-Time Controlled Text Generation with Experts and Anti-Expertscited by 441
38 more
  1. 2020GeDi: Generative Discriminator Guided Sequence Generationcited by 378
  2. 2021Self-Diagnosis and Self-Debiasing: A Proposal for Reducing Corpus-Based Bias in NLPcited by 337
  3. 2023Principled Reinforcement Learning with Human Feedback from Pairwise or K-wise Comparisonscited by 247
  4. 2017Lexically Constrained Decoding for Sequence Generation Using Grid Beam Searchcited by 237
  5. 2023Efficient Guided Generation for Large Language Modelscited by 200
  6. 2023Grammar-Constrained Decoding for Structured NLP Tasks without Finetuningcited by 197
  7. 2024Preference Fine-Tuning of LLMs Should Leverage Suboptimal, On-Policy Datacited by 185
  8. 2018Fast Lexically Constrained Decoding with Dynamic Beam Allocation for Neural Machine Translationcited by 182
  9. 2022COLD Decoding: Energy-based Constrained Text Generation with Langevin Dynamicscited by 161
  10. 2022Synchromesh: Reliable code generation from pre-trained language modelscited by 156
  11. 2021NeuroLogic A*esque Decoding: Constrained Text Generation with Lookahead Heuristicscited by 154
  12. 2024Turning Up the Heat: Min-p Sampling for Creative and Coherent LLM Outputscited by 150
  13. 2020NeuroLogic Decoding: (Un)supervised Neural Text Generation with Predicate Logic Constraintscited by 149
  14. 2022RL with KL penalties is better viewed as Bayesian inferencecited by 135
  15. 2018Learning to Write with Cooperative Discriminatorscited by 119
  16. 2023FollowBench: A Multi-level Fine-grained Constraints Following Benchmark for Large Language Modelscited by 117
  17. 2024Fast Best-of-N Decoding via Speculative Rejectioncited by 112
  18. 2022Prompting Is Programming: A Query Language for Large Language Modelscited by 110
  19. 2016Guided Open Vocabulary Image Captioning with Constrained Beam Searchcited by 105
  20. 2020Einsum Networks: Fast and Scalable Learning of Tractable Probabilistic Circuitscited by 103
  21. 2022Semantic Probabilistic Layers for Neuro-Symbolic Learningcited by 103
  22. 2024Guiding LLMs The Right Way: Fast, Non-Invasive Constrained Generationcited by 101
  23. 2018CGMH: Constrained Sentence Generation by Metropolis-Hastings Samplingcited by 94
  24. 2024Probabilistic Inference in Language Models via Twisted Sequential Monte Carlocited by 90
  25. 2024Watch Every Step! LLM Agent Learning via Iterative Step-Level Process Refinementcited by 88
  26. 2023Controlled Text Generation with Natural Language Instructionscited by 87
  27. 2023Sequential Monte Carlo Steering of Large Language Models using Probabilistic Programscited by 80
  28. 2025Organize the Web: Constructing Domains Enhances Pre-Training Data Curationcited by 76
  29. 2023Evaluating Large Language Models on Controlled Generation Taskscited by 73
  30. 2022RankGen: Improving Text Generation with Large Ranking Modelscited by 72
  31. 2025JSONSchemaBench: A Rigorous Benchmark of Structured Outputs for Language Modelscited by 71
  32. 2022Gradient-Based Constrained Sampling from Language Modelscited by 64
  33. 2023Controlled Text Generation via Language Model Arithmeticcited by 51
  34. 2024Variational Best-of-N Alignmentcited by 51
  35. 2024Automata-based constraints for language model decodingcited by 51
  36. 2020Data Boost: Text Data Augmentation Through Reinforcement Learning Guided Conditional Generationcited by 47
  37. 2025Syntactic and Semantic Control of Large Language Models via Sequential Monte Carlocited by 46
  38. 2025Evaluation of Best-of-N Sampling Strategies for Language Model Alignmentcited by 39

50 most cited of 60 All 60 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 3; 2026: 1.

4 papers.

  1. 2025Neurosymbolic Diffusion Modelscited by 13
  2. 2025RefineStat: Efficient Exploration for Probabilistic Program Synthesiscited by 5
  3. 2026Entropy-Aligned Decoding of LMs for Better Writing and Reasoningcited by 1
  4. 2025Learning Tractable Distributions Of Language Model Continuationscited by 0