mlsanityData to 14 Oct 2026

Grammar-Constrained Decoding for Structured NLP Tasks without Finetuning

arXiv 2305.13971 ↗First posted 23 May 2023cs.CL cs.AI cs.LGcited by 197

Not on the map. Why?

Cited by 25 in LLM agents and tool use, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Despite their impressive performance, large language models (LMs) still struggle with reliably generating complex output structures when not finetuned to follow the required output format exactly. To address this issue, grammar-constrained decoding (GCD) can be used to control the generation of LMs, guaranteeing that the output follows a given structure. Most existing GCD methods are, however, limited to specific tasks, such as parsing or code generation. In this work, we demonstrate that formal grammars can describe the output space for a much wider range of tasks and argue that GCD can serve as a unified framework for structured NLP tasks in general. For increased flexibility, we introduce input-dependent grammars, which allow the grammar to depend on the input and thus enable the generation of different output structures for different inputs. We then empirically demonstrate the power and flexibility of GCD-enhanced LMs on (1) information extraction, (2) entity disambiguation, and (3) constituency parsing. Our results indicate that grammar-constrained LMs substantially outperform unconstrained LMs or even beat task-specific finetuned models. Grammar constraints thus hold great promise for harnessing off-the-shelf LMs for a wide range of structured NLP tasks, especially where training data is scarce or finetuning is expensive. Code and data: https://github.com/epfl-dlab/GCD.

Key facts. Cited by 197 papers in the map (data through 14 Oct 2026). Builds on 25 of its 49 references that are in the map. First posted on arXiv 23 May 2023.

Builds on

25 of 49 references.

References · 25

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  3. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  4. 2015Neural Machine Translation of Rare Words with Subword Unitscited by 2,918
  5. 2020Exploiting Cloze Questions for Few Shot Text Classification and Natural Language Inferencecited by 938
  6. 2021PICARD: Parsing Incrementally for Constrained Auto-Regressive Decoding from Language Modelscited by 411
  7. 2020Autoregressive Entity Retrievalcited by 334
  8. 2017Lexically Constrained Decoding for Sequence Generation Using Grid Beam Searchcited by 237
  9. 2018Constituency Parsing with a Self-Attentive Encodercited by 199
  10. 2021Structured Prediction as Translation between Augmented Natural Languagescited by 183
  11. 2018Fast Lexically Constrained Decoding with Dynamic Beam Allocation for Neural Machine Translationcited by 182
  12. 2021Constrained Language Models Yield Few-Shot Semantic Parserscited by 163
13 more
  1. 2019On NMT Search Errors and Model Errors: Cat Got Your Tongue?cited by 114
  2. 2016Recurrent Neural Network Grammarscited by 109
  3. 2016Guided Open Vocabulary Image Captioning with Constrained Beam Searchcited by 105
  4. 2023Exploiting Asymmetry for Synthetic Training Data Generation: SynthIE and the Case of Information Extractioncited by 89
  5. 2022Thinking about GPT-3 In-Context Learning for Biomedical IE? Think Againcited by 83
  6. 2017Deep Joint Entity Disambiguation with Local Neural Attentioncited by 79
  7. 2022ReFinED: An Efficient Zero-shot-capable Approach to End-to-End Entity Linkingcited by 74
  8. 2014Grammar as a Foreign Languagecited by 70
  9. 2018Improving Entity Linking by Modeling Latent Relations between Mentionscited by 53
  10. 2021GenIE: Generative Information Extractioncited by 53
  11. 2020Fast and Accurate Neural CRF Constituency Parsingcited by 50
  12. 2022Autoregressive Structured Prediction with Language Modelscited by 43
  13. 2020Augmented Natural Language for Generative Sequence Labelingcited by 29

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 4; 2024: 31; 2025: 53; 2026: 109.

197 papers.

Sort:

By area:

  1. 2025Securing AI Agents with Information-Flow Controlcited by 121
  2. 2024Guiding LLMs The Right Way: Fast, Non-Invasive Constrained Generationcited by 101
  3. 2023GLiNER: Generalist Model for Named Entity Recognition using Bidirectional Transformercited by 88
  4. 2024Perception Tokens Enhance Visual Reasoning in Multimodal Language Modelscited by 71
  5. 2025JSONSchemaBench: A Rigorous Benchmark of Structured Outputs for Language Modelscited by 71
  6. 2024SynCode: LLM Generation with Grammar Augmentationcited by 69
  7. 2024Grammar-Aligned Decodingcited by 54
  8. 2024Automata-based constraints for language model decodingcited by 51
  9. 2025Syntactic and Semantic Control of Large Language Models via Sequential Monte Carlocited by 46
  10. 2023KCTS: Knowledge-Constrained Tree Search Decoding with Token-Level Hallucination Detectioncited by 45
  11. 2025Flexible and Efficient Grammar-Constrained Decodingcited by 43
  12. 2024Formal-LLM: Integrating Formal Language and Natural Language for Controllable LLM-based Agentscited by 40
38 more
  1. 2024From Language Models over Tokens to Language Models over Characterscited by 31
  2. 2024IterGen: Iterative Semantic-aware Structured LLM Generation with Backtrackingcited by 29
  3. 2025Learning to Generate Structured Output with Schema Reinforcement Learningcited by 29
  4. 2024Octopus: On-device language model for function calling of software APIscited by 27
  5. 2024StructuredRAG: JSON Response Formatting with Large Language Modelscited by 22
  6. 2024To the Globe (TTG): Towards Language-Driven Guaranteed Travel Planningcited by 21
  7. 2025Fast Controlled Generation from Language Models with Adaptive Weighted Rejection Samplingcited by 21
  8. 2025Constrained Discrete Diffusioncited by 17
  9. 2025Unveiling the Potential of Diffusion Large Language Model in Controllable Generationcited by 15
  10. 2025Constrained Sampling for Language Models Should Be Easy: An MCMC Perspectivecited by 14
  11. 2023Don't Fine-Tune, Decode: Syntax Error-Free Tool Use via Constrained Decodingcited by 13
  12. 2026Group of Skills: Group-Structured Skill Retrieval for Agent Skill Librariescited by 11
  13. 2024Comparing GPT-4 and Open-Source Language Models in Misinformation Mitigationcited by 9
  14. 2024Sketch-Guided Constrained Decoding for Boosting Blackbox Large Language Models without Logit Accesscited by 9
  15. 2024Grammar-based Game Description Generation using Large Language Modelscited by 7
  16. 2024A Simple yet Effective Training-free Prompt-free Approach to Chinese Spelling Correction Based on Large Language Modelscited by 7
  17. 2025Shh, don't say that! Domain Certification in LLMscited by 7
  18. 2025GraphRAFT: Retrieval Augmented Fine-Tuning for Knowledge Graphs on Graph Databasescited by 7
  19. 2026Lookahead-then-Verify: Reliable Constrained Decoding for Diffusion LLMs under Context-Free Grammarscited by 7
  20. 2026this-that-model-1.0: A typed decision model that decides in 30 ms, for a millionth of a centcited by 7
  21. 2023JsonTuning: Towards Generalizable, Robust, and Controllable Instruction Tuningcited by 6
  22. 2024Synthetic Programming Elicitation for Text-to-Code in Very Low-Resource Programming and Formal Languagescited by 6
  23. 2025ADAPT: Actively Discovering and Adapting to Preferences for any Taskcited by 6
  24. 2024An Autoregressive Text-to-Graph Framework for Joint Entity and Relation Extractioncited by 5
  25. 2024FANTAstic SEquences and Where to Find Them: Faithful and Efficient API Call Generation through State-tracked Constrained Decoding and Rerankingcited by 5
  26. 2024Using a Feedback Loop for LLM-based Infrastructure as Code Generationcited by 5
  27. 2025Efficient and Asymptotically Unbiased Constrained Decoding for Large Language Modelscited by 5
  28. 2025Pel, A Programming Language for Orchestrating AI Agentscited by 5
  29. 2025TreeCoder: Systematic Exploration and Optimisation of Decoding and Constraints for LLM Code Generationcited by 5
  30. 2026Thinking Before Constraining: A Unified Decoding Framework for Large Language Modelscited by 5
  31. 2024Guiding Large Language Models to Generate Computer-Parsable Contentcited by 4
  32. 2024Entity Decomposition with Filtering: A Zero-Shot Clinical Named Entity Recognition Frameworkcited by 4
  33. 2025Leveraging Vision-Language Models for Open-Vocabulary Instance Segmentation and Trackingcited by 4
  34. 2025Semantic Probabilistic Control of Language Modelscited by 4
  35. 2025Verifiable Natural Language to Linear Temporal Logic Translation: A Benchmark Dataset and Evaluation Suitecited by 4
  36. 2025Mind and Motion Aligned: A Joint Evaluation IsaacSim Benchmark for Task Planning and Low-Level Policies in Mobile Manipulationcited by 4
  37. 2025XML Prompting as Grammar-Constrained Interaction: Fixed-Point Semantics, Convergence Guarantees, and Human-AI Protocolscited by 4
  38. 2025How to Make Large Language Models Generate 100% Valid Molecules?cited by 4

50 most cited of 197 All 197 (JSON)