mlsanityData to 14 Oct 2026

Guiding LLMs The Right Way: Fast, Non-Invasive Constrained Generation

arXiv 2403.06988 ↗First posted 7 Feb 2024cs.LG cs.CLcited by 101

Not on the map. Why?

Abstract (arXiv)

To ensure that text generated by large language models (LLMs) is in an expected format, constrained decoding proposes to enforce strict formal language constraints during generation. However, as we show in this work, not only do such methods incur performance overhead during generation, but many of them also significantly impair task accuracy, if they do not correctly align the underlying LLM sub-word vocabularies with external constraints. To address this, we present a novel decoding algorithm, DOMINO, that can enforce constraints in a fully subword-aligned fashion, while leveraging pre-computation and speculative decoding to achieve virtually no overhead and in some cases even almost 2× speedup over unconstrained decoding -- thereby outperforming existing approaches by a wide margin.

Key facts. Cited by 101 papers in the map (data through 14 Oct 2026). Builds on 17 of its 25 references that are in the map. First posted on arXiv 7 Feb 2024.

Builds on

17 of 25 references.

References · 17

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2023GPT-4 Technical Reportcited by 18,396
  4. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  5. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 13,028
  6. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  7. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  8. 2023Mistral 7Bcited by 3,164
  9. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,918
  10. 2019HuggingFace's Transformers: State-of-the-art Natural Language Processingcited by 2,218
  11. 2018SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processingcited by 1,989
  12. 2024Mixtral of Expertscited by 1,823
5 more
  1. 2023Accelerating Large Language Model Decoding with Speculative Samplingcited by 1,006
  2. 2021PICARD: Parsing Incrementally for Constrained Auto-Regressive Decoding from Language Modelscited by 411
  3. 2023Efficient Guided Generation for Large Language Modelscited by 200
  4. 2023Grammar-Constrained Decoding for Structured NLP Tasks without Finetuningcited by 197
  5. 2022Synchromesh: Reliable code generation from pre-trained language modelscited by 156

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 16; 2025: 40; 2026: 45.

101 papers.

Sort:

By area:

  1. 2025Securing AI Agents with Information-Flow Controlcited by 121
  2. 2024Formal Mathematical Reasoning: A New Frontier in AIcited by 81
  3. 2025Design Patterns for Securing LLM Agents against Prompt Injectionscited by 73
  4. 2024Perception Tokens Enhance Visual Reasoning in Multimodal Language Modelscited by 71
  5. 2024SynCode: LLM Generation with Grammar Augmentationcited by 69
  6. 2025Flexible and Efficient Grammar-Constrained Decodingcited by 43
  7. 2024Segment Any Text: A Universal Approach for Robust, Efficient and Adaptable Sentence Segmentationcited by 41
  8. 2024Moral Alignment for LLM Agentscited by 39
  9. 2025Type-Constrained Code Generation with Language Modelscited by 35
  10. 2024On the Challenges and Opportunities in Generative AIcited by 31
  11. 2024StructuredRAG: JSON Response Formatting with Large Language Modelscited by 22
  12. 2025Fast Controlled Generation from Language Models with Adaptive Weighted Rejection Samplingcited by 21
38 more
  1. 2025Spotlight Your Instructions: Instruction-following with Dynamic Attention Steeringcited by 21
  2. 2025Reliable Text-to-SQL with Adaptive Abstentioncited by 20
  3. 2025SLOT: Structuring the Output of Large Language Modelscited by 18
  4. 2024CATP-LLM: Empowering Large Language Models for Cost-Aware Tool Planningcited by 14
  5. 2025Constrained Decoding of Diffusion LLMs with Context-Free Grammarscited by 14
  6. 2025Pie: A Programmable Serving System for Emerging LLM Applicationscited by 14
  7. 2025SafeSteer: Interpretable Safety Steering with Refusal-Evasion in LLMscited by 10
  8. 2025Decoupling Task-Solving and Output Formatting in LLM Generationcited by 9
  9. 2024Reflections from the 2024 Large Language Model (LLM) Hackathon for Applications in Materials Science and Chemistrycited by 8
  10. 2024Breaking the Hourglass Phenomenon of Residual Quantization: Enhancing the Upper Bound of Generative Retrievalcited by 7
  11. 2025GraphRAFT: Retrieval Augmented Fine-Tuning for Knowledge Graphs on Graph Databasescited by 7
  12. 2025Sampling from Your Language Model One Byte at a Timecited by 7
  13. 2025Empowering Healthcare Practitioners with Language Models: Structuring Speech Transcripts in Two Real-World Clinical Applicationscited by 7
  14. 2026Lookahead-then-Verify: Reliable Constrained Decoding for Diffusion LLMs under Context-Free Grammarscited by 7
  15. 2024Constrained Decoding with Speculative Lookaheadscited by 6
  16. 2025CallNavi, A Challenge and Empirical Study on LLM Function Calling and Routingcited by 6
  17. 2026A Survey on Evaluating Quality and Trustworthiness in LLM-Generated Datacited by 6
  18. 2026Neuro-Symbolic Synergy for World Modelingcited by 6
  19. 2025Efficient and Asymptotically Unbiased Constrained Decoding for Large Language Modelscited by 5
  20. 2025Pel, A Programming Language for Orchestrating AI Agentscited by 5
  21. 2025OpeNLGauge: An Explainable Metric for NLG Evaluation with Open-Weights LLMscited by 4
  22. 2025Leveraging Vision-Language Models for Open-Vocabulary Instance Segmentation and Trackingcited by 4
  23. 2025Semantic Probabilistic Control of Language Modelscited by 4
  24. 2024Approximately Aligned Decodingcited by 3
  25. 2025Earley-Driven Dynamic Pruning for Efficient Structured Decodingcited by 3
  26. 2026Typed Decision Models: An Early Evidence Audit and Evaluation Checklistcited by 3
  27. 2024Structured Object Language Modeling (SoLM): Native Structured Objects Generation Conforming to Complex Schemas with Self-Supervised Denoisingcited by 2
  28. 2025Every Software as an Agent: Blueprint and Case Studycited by 2
  29. 2025A City of Millions: Mapping Literary Social Networks At Scalecited by 2
  30. 2025HyGenar: An LLM-Driven Hybrid Genetic Algorithm for Few-Shot Grammar Generationcited by 2
  31. 2025Making Logic a First-Class Citizen in Generative ML for Networkingcited by 2
  32. 2025Opponent Shaping in LLM Agentscited by 2
  33. 2025BEAVER: An Efficient Deterministic LLM Verifiercited by 2
  34. 2026Schema-Key Wording as an Instruction Channel in Structured Generation under Constrained Decodingcited by 2
  35. 2026The Extrapolation Cliff in On-Policy Distillation of Near-Deterministic Structured Outputscited by 2
  36. 2026Actionable Hallucination Detection: Translating Latent Uncertainty into Agentic Critiquecited by 2
  37. 2024Bel Esprit: Multi-Agent Framework for Building AI Model Pipelinescited by 1
  38. 2025Lost in Space: Finding the Right Tokens for Structured Outputcited by 1

50 most cited of 101 All 101 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Large-Scale Constraint Generation -- Can LLMs Parse Hundreds of Constraints?
  2. 2026Thinking Before Constraining: A Unified Decoding Framework for Large Language Models
  3. 2024Direct Alignment of Draft Model for Speculative Decoding with Chat-Fine-Tuned LLMs
  4. 2025Accelerating LLM Inference with Lossless Speculative Decoding Algorithms for Heterogeneous Vocabularies
  5. 2024Decoding at the Speed of Thought: Harnessing Parallel Decoding of Lexical Units for LLMs
  6. 2024A Comprehensive Survey of Accelerated Generation Techniques in Large Language Models
  7. 2026Constrained Decoding Eliminates Structural Failures in Small LLMs but Reveals a Scale-Dependent Semantic Gap
  8. 2024A Thorough Examination of Decoding Methods in the Era of LLMs