mlsanityData to 14 Oct 2026

Sequential Monte Carlo Steering of Large Language Models using Probabilistic Programs

arXiv 2306.03081 ↗First posted 5 Jun 2023cs.AI cs.CL cs.PLcited by 80

Not on the map. Why?

Abstract (arXiv)

Even after fine-tuning and reinforcement learning, large language models (LLMs) can be difficult, if not impossible, to control reliably with prompts alone. We propose a new inference-time approach to enforcing syntactic and semantic constraints on the outputs of LLMs, called sequential Monte Carlo (SMC) steering. The key idea is to specify language generation tasks as posterior inference problems in a class of discrete probabilistic sequence models, and replace standard decoding with sequential Monte Carlo inference. For a computational cost similar to that of beam search, SMC can steer LLMs to solve diverse tasks, including infilling, generation under syntactic constraints, and prompt intersection. To facilitate experimentation with SMC steering, we present a probabilistic programming library, LLaMPPL (https://github.com/probcomp/hfppl), for concisely specifying new generation tasks as language model probabilistic programs, and automating steering of LLaMA-family Transformers.

Key facts. Cited by 80 papers in the map (data through 14 Oct 2026). Builds on 25 of its 39 references that are in the map. First posted on arXiv 5 Jun 2023.

Builds on

25 of 39 references.

References · 25

  1. 2022Training language models to follow instructions with human feedbackcited by 15,971
  2. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  3. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  4. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  5. 2022Large Language Models Are Human-Level Prompt Engineerscited by 1,045
  6. 2019Plug and Play Language Models: A Simple Approach to Controlled Text Generationcited by 799
  7. 2016Diverse Beam Search: Decoding Diverse Solutions from Neural Sequence Modelscited by 301
  8. 2022Efficient Training of Language Models to Fill in the Middlecited by 255
  9. 2023Planning with Large Language Models for Code Generationcited by 182
  10. 2022COLD Decoding: Energy-based Constrained Text Generation with Langevin Dynamicscited by 161
  11. 2022Synchromesh: Reliable code generation from pre-trained language modelscited by 156
  12. 2021NeuroLogic A*esque Decoding: Constrained Text Generation with Lookahead Heuristicscited by 154
13 more
  1. 2022RL with KL penalties is better viewed as Bayesian inferencecited by 135
  2. 2020Enabling Language Models to Fill in the Blankscited by 122
  3. 2018Correcting Length Bias in Neural Machine Translationcited by 120
  4. 2020A Distributional Approach to Controlled Text Generationcited by 119
  5. 2020If beam search is the answer, what was the question?cited by 116
  6. 2022Prompting Is Programming: A Query Language for Large Language Modelscited by 110
  7. 2022Language Model Cascadescited by 107
  8. 2018CGMH: Constrained Sentence Generation by Metropolis-Hastings Samplingcited by 94
  9. 2021Controlled Text Generation as Continuous Optimization with Multiple Constraintscited by 70
  10. 2023Tractable Control for Autoregressive Language Generationcited by 65
  11. 2020Directed Beam Search: Plug-and-Play Lexically Constrained Language Generationcited by 12
  12. 2023LaMPP: Language Models as Probabilistic Priors for Perception and Actioncited by 11
  13. 2022Recursive Monte Carlo and Variational Inference with Auxiliary Variablescited by 8

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 5; 2024: 19; 2025: 28; 2026: 28.

80 papers.

Sort:

By area:

  1. 2025A General Framework for Inference-time Scaling and Steering of Diffusion Modelscited by 224
  2. 2023Efficient Guided Generation for Large Language Modelscited by 200
  3. 2024Derivative-Free Guidance in Continuous and Discrete Diffusion Models with Soft Value-Based Decodingcited by 136
  4. 2023From Word Models to World Models: Translating from Natural Language to the Probabilistic Language of Thoughtcited by 119
  5. 2023Amortizing intractable inference in large language modelscited by 97
  6. 2024Probabilistic Inference in Language Models via Twisted Sequential Monte Carlocited by 90
  7. 2025Inference-Time Alignment in Diffusion Models with Reward-Guided Generation: Tutorial and Reviewcited by 88
  8. 2024Automata-based constraints for language model decodingcited by 51
  9. 2025Syntactic and Semantic Control of Large Language Models via Sequential Monte Carlocited by 46
  10. 2024Space Group Informed Transformer for Crystalline Materials Generationcited by 41
  11. 2025Enforcing Temporal Constraints for LLM Agentscited by 34
  12. 2024Adaptable Logical Control for Large Language Modelscited by 30
38 more
  1. 2025Hypothesis-Driven Theory-of-Mind Reasoning for Large Language Modelscited by 29
  2. 2024Pragmatic Instruction Following and Goal Assistance via Cooperative Language-Guided Inverse Planningcited by 28
  3. 2024Doing Experiments and Revising Rules with Natural Language and Probabilistic Reasoningcited by 24
  4. 2024GAVEL: Generating Games Via Evolution and Language Modelscited by 24
  5. 2023The Neuro-Symbolic Inverse Planning Engine (NIPE): Modeling Probabilistic Social Inferences from Linguistic Inputscited by 23
  6. 2024Step-by-Step Reasoning for Math Problems via Twisted Sequential Monte Carlocited by 22
  7. 2025Fast Controlled Generation from Language Models with Adaptive Weighted Rejection Samplingcited by 21
  8. 2025Rollout Roulette: A Probabilistic Inference Approach to Inference-Time Scaling of LLMs using Particle-Based Monte Carlo Methodscited by 20
  9. 2025Composition and Control with Distilled Energy Diffusion Models and Sequential Monte Carlocited by 20
  10. 2024Estimating the Probabilities of Rare Outputs in Language Modelscited by 16
  11. 2025DINGO: Constrained Inference for Diffusion LLMscited by 16
  12. 2025Constrained Sampling for Language Models Should Be Easy: An MCMC Perspectivecited by 14
  13. 2024Can a Bayesian Oracle Prevent Harm from an Agent?cited by 13
  14. 2025Self-Steering Language Modelscited by 13
  15. 2025Neurosymbolic Diffusion Modelscited by 13
  16. 2024Loose LIPS Sink Ships: Asking Questions in Battleship with Language-Informed Program Samplingcited by 12
  17. 2024Controllable Generation via Locally Constrained Resamplingcited by 11
  18. 2024Amortized Control of Continuous State Space Feynman-Kac Model for Irregular Time Seriescited by 10
  19. 2024Latent Logic Tree Extraction for Event Sequence Explanation from LLMscited by 9
  20. 2026Rare Event Analysis of Large Language Modelscited by 8
  21. 2025Inference-Time Scaling of Diffusion Language Models via Trajectory Refinementcited by 7
  22. 2024Understanding Epistemic Language with a Language-augmented Bayesian Theory of Mindcited by 6
  23. 2025Reinforced sequential Monte Carlo for amortised samplingcited by 6
  24. 2026What's the plan? Metrics for implicit planning in LLMs and their application to rhyme generation and question answeringcited by 6
  25. 2024Local and Global Decoding in Text Generationcited by 5
  26. 2026Transducing Language Modelscited by 5
  27. 2026Sampling for Quality: Training-Free Reward-Guided LLM Decoding via Sequential Monte Carlocited by 5
  28. 2025Semantic Probabilistic Control of Language Modelscited by 4
  29. 2025Mitigating Premature Exploitation in Particle-based Monte Carlo for Inference-Time Scalingcited by 4
  30. 2024Approximately Aligned Decodingcited by 3
  31. 2025Syntactic Control of Language Models by Posterior Inferencecited by 3
  32. 2025Constrained Adaptive Rejection Samplingcited by 3
  33. 2025Anytime-Valid Answer Sufficiency Certificates for LLM Generation via Sequential Information Liftcited by 3
  34. 2026Ensembling Language Models with Sequential Monte Carlocited by 3
  35. 2026Reject, Resample, Repeat: Understanding Parallel Reasoning in Language Model Inferencecited by 3
  36. 2024Multi-Layer Ranking with Large Language Models for News Source Recommendationcited by 2
  37. 2025Improving Constrained Language Generation via Self-Distilled Twisted Sequential Monte Carlocited by 2
  38. 2025Controlling Multimodal LLMs via Reward-guided Decodingcited by 2

50 most cited of 80 All 80 (JSON)