mlsanityData to 14 Oct 2026

DINGO: Constrained Inference for Diffusion LLMs

arXiv 2505.23061 ↗First posted 29 May 2025cs.LG cs.PL cs.SEcited by 16

Not on the map. Why?

Abstract (arXiv)

Diffusion LLMs have emerged as a promising alternative to conventional autoregressive LLMs, offering significant potential for improved runtime efficiency. However, existing diffusion models lack the ability to provably enforce user-specified formal constraints, such as regular expressions, which makes them unreliable for tasks that require structured outputs, such as fixed-schema JSON generation. Unlike autoregressive models that generate tokens sequentially, diffusion LLMs predict a block of tokens in parallel. This parallelism makes traditional constrained decoding algorithms, which are designed for sequential token prediction, ineffective at preserving the true output distribution. To address this limitation, we propose DINGO, a dynamic programming-based constrained decoding strategy that is both efficient and provably distribution-preserving. DINGO enables sampling of output strings with the highest probability under the model's predicted distribution, while strictly satisfying any user-specified regular expression. On standard symbolic math and JSON generation benchmarks, DINGO achieves up to a 68 percentage point improvement over unconstrained inference

Key facts. Cited by 16 papers in the map (data through 14 Oct 2026). Builds on 20 of its 44 references that are in the map. First posted on arXiv 29 May 2025.

Builds on

20 of 44 references.

References · 20

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  2. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  3. 2021Structured Denoising Diffusion Models in Discrete State-Spacescited by 1,719
  4. 2025Large Language Diffusion Modelscited by 912
  5. 2024GSM-Symbolic: Understanding the Limitations of Mathematical Reasoning in Large Language Modelscited by 550
  6. 2023LeanDojo: Theorem Proving with Retrieval-Augmented Language Modelscited by 427
  7. 2025Block Diffusion: Interpolating Between Autoregressive and Diffusion Language Modelscited by 409
  8. 2023Logic-LM: Empowering Large Language Models with Symbolic Solvers for Faithful Logical Reasoningcited by 391
  9. 2023Efficient Guided Generation for Large Language Modelscited by 200
  10. 2022SSD-LM: Semi-autoregressive Simplex-based Diffusion Language Model for Text Generation and Modular Controlcited by 194
  11. 2025d1: Scaling Reasoning in Diffusion Large Language Models via Reinforcement Learningcited by 186
  12. 2022Synchromesh: Reliable code generation from pre-trained language modelscited by 156
8 more
  1. 2023Sequential Monte Carlo Steering of Large Language Models using Probabilistic Programscited by 80
  2. 2024XGrammar: Flexible and Efficient Structured Generation Engine for Large Language Modelscited by 75
  3. 2024SynCode: LLM Generation with Grammar Augmentationcited by 69
  4. 2024Grammar-Aligned Decodingcited by 54
  5. 2024Automata-based constraints for language model decodingcited by 51
  6. 2025Syntactic and Semantic Control of Large Language Models via Sequential Monte Carlocited by 46
  7. 2025CRANE: Reasoning with constrained LLM generationcited by 38
  8. 2022Validating Large Language Models with ReLMcited by 33

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 3; 2026: 13.

16 papers.

Sort:

By area:

  1. 2025A Survey on Diffusion Language Modelscited by 78
  2. 2025Constrained Decoding of Diffusion LLMs with Context-Free Grammarscited by 14
  3. 2026SEVerA: Verified Synthesis of Self-Evolving Agentscited by 8
  4. 2026Lookahead-then-Verify: Reliable Constrained Decoding for Diffusion LLMs under Context-Free Grammarscited by 7
  5. 2026Constrained Decoding for Diffusion Language Models via Efficient Inference over Finite Automatacited by 4
  6. 2026Constraint Tax in Open-Weight LLMs: An Empirical Study of Tool Calling Suppression Under Structured Output Constraintscited by 3
  7. 2026AnCoder: Anchored Code Generation via Discrete Diffusion Modelscited by 1
  8. 2026The Architecture of Errors: From Universal Impossibility to Patch-Local LLM Reliabilitycited by 1
  9. 2026EPIC: Efficient and Parallel Inference under CFG Constraints for Diffusion Language Modelscited by 1
  10. 2026Accelerating Masked Diffusion Large Language Models: A Survey of Efficient Inference Techniquescited by 1
  11. 2025Constraints-Guided Diffusion Reasoner for Neuro-Symbolic Learningcited by 0
  12. 2026Imprompt: A Language Framework for Prompt Programmingcited by 0
4 more
  1. 2026Constraints Are Graphs, Not Chains: Exact Decoding for Diffusion Language Modelscited by 0
  2. 2026Twist, Don't Tilt: Trajectory-Exact Constrained Decoding for Masked Diffusion Modelscited by 0
  3. 2026Grab a Coffee: Future-Aware Guidance for Discrete Diffusion with Compiled Objectivescited by 0
  4. 2026Intent-Triggered Anchoring for Multi-Token Format Units in Parallel Decoding of Diffusion Language Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Unlocking Lossless Speedups in LLMs via Discrete Diffusion
  2. 2025Accelerating Diffusion LLM Inference via Local Determinism Propagation
  3. 2026Continuous Diffusion Models Can Obey Formal Syntax
  4. 2025Accelerating Diffusion LLMs via Adaptive Parallel Decoding
  5. 2025Diffusion Language Model Inference with Monte Carlo Tree Search
  6. 2025ParallelBench: Understanding the Trade-offs of Parallel Decoding in Diffusion LLMs
  7. 2025Diffusion LLMs Can Do Faster-Than-AR Inference via Discrete Diffusion Forcing
  8. 2025Diffusion Language Models Know the Answer Before Decoding