mlsanityData to 14 Oct 2026

Grammar Prompting for Domain-Specific Language Generation with Large Language Models

arXiv 2305.19234 ↗First posted 30 May 2023cs.CL cs.AIcited by 58

Not on the map. Why?

Abstract (arXiv)

Large language models (LLMs) can learn to perform a wide range of natural language tasks from just a handful of in-context examples. However, for generating strings from highly structured languages (e.g., semantic parsing to complex domain-specific languages), it is challenging for the LLM to generalize from just a few exemplars. We propose grammar prompting, a simple approach to enable LLMs to use external knowledge and domain-specific constraints, expressed through a grammar in Backus--Naur Form (BNF), during in-context learning. Grammar prompting augments each demonstration example with a specialized grammar that is minimally sufficient for generating the particular output example, where the specialized grammar is a subset of the full DSL grammar. For inference, the LLM first predicts a BNF grammar given a test input, and then generates the output according to the rules of the grammar. Experiments demonstrate that grammar prompting can enable LLMs to perform competitively on a diverse set of DSL generation tasks, including semantic parsing (SMCalFlow, Overnight, GeoQuery), PDDL planning, and SMILES-based molecule generation.

Key facts. Cited by 58 papers in the map (data through 14 Oct 2026). Builds on 75 of its 100 references that are in the map. First posted on arXiv 30 May 2023.

Builds on

75 of 100 references.

References · 75

  1. 2020Language Models are Few-Shot Learnerscited by 32,419
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  3. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  4. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  5. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
  6. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,182
  7. 2021Finetuned Language Models Are Zero-Shot Learnerscited by 3,780
  8. 2021Program Synthesis with Large Language Modelscited by 3,637
  9. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,211
  10. 2022Do As I Can, Not As I Say: Grounding Language in Robotic Affordancescited by 2,348
  11. 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,191
  12. 2022Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Themcited by 1,807
38 more
  1. 2021Multitask Prompted Training Enables Zero-Shot Task Generalizationcited by 1,470
  2. 2023PaLM 2 Technical Reportcited by 1,162
  3. 2022Program of Thoughts Prompting: Disentangling Computation from Reasoning for Numerical Reasoning Taskscited by 1,076
  4. 2022Language Models as Zero-Shot Planners: Extracting Actionable Knowledge for Embodied Agentscited by 1,035
  5. 2021Show Your Work: Scratchpads for Intermediate Computation with Language Modelscited by 886
  6. 2023Visual ChatGPT: Talking, Drawing and Editing with Visual Foundation Modelscited by 668
  7. 2018Junction Tree Variational Autoencoder for Molecular Graph Generationcited by 652
  8. 2023ViperGPT: Visual Inference via Python Execution for Reasoningcited by 603
  9. 2023MM-REACT: Prompting ChatGPT for Multimodal Reasoning and Actioncited by 517
  10. 2023LLM+P: Empowering Large Language Models with Optimal Planning Proficiencycited by 501
  11. 2021PICARD: Parsing Incrementally for Constrained Auto-Regressive Decoding from Language Modelscited by 411
  12. 2023Logic-LM: Empowering Large Language Models with Symbolic Solvers for Faithful Logical Reasoningcited by 391
  13. 2022PlanBench: An Extensible Benchmark for Evaluating Large Language Models on Planning and Reasoning about Changecited by 382
  14. 2022A Systematic Evaluation of Large Language Models of Codecited by 363
  15. 2023Tool Learning with Foundation Modelscited by 302
  16. 2023Faithful Chain-of-Thought Reasoningcited by 288
  17. 2017Grammar Variational Autoencodercited by 285
  18. 2023ART: Automatic multi-step reasoning and tool-use for large language modelscited by 197
  19. 2023TaskMatrix.AI: Completing Tasks by Connecting Foundation Models with Millions of APIscited by 184
  20. 2022Translation between Molecules and Natural Languagecited by 177
  21. 2021Constrained Language Models Yield Few-Shot Semantic Parserscited by 163
  22. 2022Synchromesh: Reliable code generation from pre-trained language modelscited by 156
  23. 2018PDDLStream: Integrating Symbolic Planners and Blackbox Samplers via Optimistic Adaptive Planningcited by 150
  24. 2020Compositional Generalization and Natural Language Variation: Can a Semantic Parsing Approach Handle Both?cited by 147
  25. 2023Compositional Exemplars for In-context Learningcited by 144
  26. 2022Diverse Demonstrations Improve In-context Compositional Generalizationcited by 139
  27. 2022ChemBERTa-2: Towards Chemical Foundation Modelscited by 127
  28. 2023Solving Math Word Problems by Combining Language Models With Symbolic Solverscited by 124
  29. 2020Task-Oriented Dialogue as Dataflow Synthesiscited by 122
  30. 2016Recurrent Neural Network Grammarscited by 109
  31. 2022Language Model Cascadescited by 107
  32. 2018Syntax-Directed Variational Autoencoder for Structured Datacited by 103
  33. 2023SatLM: Satisfiability-Aided Language Models Using Declarative Promptingcited by 102
  34. 2021Skill Induction and Planning with Latent Languagecited by 99
  35. 2020Span-based Semantic Parsing for Compositional Generalizationcited by 77
  36. 2019Compound Probabilistic Context-Free Grammars for Grammar Inductioncited by 76
  37. 2020Retro*: Learning Retrosynthetic Planning with Neural Guided A* Searchcited by 76
  38. 2018A Survey on Semantic Parsingcited by 62

50 most cited of 75 All 75 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 2; 2024: 24; 2025: 17; 2026: 15.

58 papers.

Sort:

By area:

  1. 2023Efficient Guided Generation for Large Language Modelscited by 200
  2. 2024Large Language Model (LLM) for Telecommunications: A Comprehensive Survey on Principles, Key Techniques, and Opportunitiescited by 83
  3. 2024XGrammar: Flexible and Efficient Structured Generation Engine for Large Language Modelscited by 75
  4. 2025JSONSchemaBench: A Rigorous Benchmark of Structured Outputs for Language Modelscited by 71
  5. 2024A Survey on LLM-based Code Generation for Low-Resource and Domain-Specific Programming Languagescited by 66
  6. 2025QuestBench: Can LLMs ask the right question to acquire information in reasoning tasks?cited by 63
  7. 2024DeAL: Decoding-time Alignment for Large Language Modelscited by 55
  8. 2025Syntactic and Semantic Control of Large Language Models via Sequential Monte Carlocited by 46
  9. 2025Flexible and Efficient Grammar-Constrained Decodingcited by 43
  10. 2024Learning Agent-based Modeling with LLM Companions: Experiences of Novices and Experts Using ChatGPT & NetLogo Chatcited by 23
  11. 2025Fast Controlled Generation from Language Models with Adaptive Weighted Rejection Samplingcited by 21
  12. 2024Large Language Models for Constructing and Optimizing Machine Learning Workflows: A Surveycited by 19
38 more
  1. 2024SymbolicAI: A framework for logic-based approaches combining generative models and solverscited by 16
  2. 2024Generating consistent PDDL domains with Large Language Modelscited by 15
  3. 2024Synthesizing Programmatic Reinforcement Learning Policies with Large Language Model Guided Searchcited by 13
  4. 2025A Survey on Inference Engines for Large Language Models: Perspectives on Optimization and Efficiencycited by 12
  5. 2024Controllable Generation via Locally Constrained Resamplingcited by 11
  6. 2024RoboMorph: Evolving Robot Morphology using Large Language Modelscited by 8
  7. 2024DataLab: A Unified Platform for LLM-Powered Business Intelligencecited by 8
  8. 2024Grammar-based Game Description Generation using Large Language Modelscited by 7
  9. 2024VeriGraph: Scene Graphs for Execution Verifiable Robot Planningcited by 7
  10. 2026XGrammar-2: Dynamic and Efficient Structured Generation Engine for Agentic LLMscited by 7
  11. 2024Synthetic Programming Elicitation for Text-to-Code in Very Low-Resource Programming and Formal Languagescited by 6
  12. 2025MoE2: Optimizing Collaborative Inference for Edge Large Language Modelscited by 6
  13. 2025Hierarchically Encapsulated Representation for Protocol Design in Self-Driving Labscited by 6
  14. 2024Guiding Large Language Models to Generate Computer-Parsable Contentcited by 4
  15. 2025LogiCase: Effective Test Case Generation from Logical Description in Competitive Programmingcited by 4
  16. 2025Robot Operation of Home Appliances by Reading User Manualscited by 4
  17. 2025How to Auto-optimize Prompts for Domain Tasks? Adaptive Prompting and Reasoning through Evolutionary Domain Knowledge Adaptationcited by 4
  18. 2023Evaluating In-Context Learning of Libraries for Code Generationcited by 3
  19. 2024Autoware.Flex: Human-Instructed Dynamically Reconfigurable Autonomous Driving Systemscited by 3
  20. 2025Constrained Adaptive Rejection Samplingcited by 3
  21. 2024TriplePlay: Enhancing Federated Learning with CLIP for Non-IID Data and Resource Efficiencycited by 2
  22. 2025Investigating Language Model Capabilities to Represent and Process Formal Knowledge: A Preliminary Study to Assist Ontology Engineeringcited by 2
  23. 2026HAVEN: Hybrid Automated Verification ENgine for UVM Testbench Synthesis with LLMscited by 2
  24. 2026Sequential Planning via Anchored Robotic Keypointscited by 2
  25. 2024Semantic Parsing with Candidate Expressions for Knowledge Base Question Answeringcited by 1
  26. 2024G2T-LLM: Graph-to-Tree Text Encoding for Molecule Generation with Fine-Tuned Large Language Modelscited by 1
  27. 2025Randomly Sampled Language Reasoning Problems Elucidate Limitations of In-Context Learningcited by 1
  28. 2025Using (Not-so) Large Language Models to Generate Simulation Models in a Formal DSL: A Study on Reaction Networkscited by 1
  29. 2025Diverse Prompts: Illuminating the Prompt Space of Large Language Models with MAP-Elitescited by 1
  30. 2026Grammar-Aware Literate Generative Mathematical Programming with Compiler-in-the-Loopcited by 1
  31. 2026Shaping Schema via Language Representation as the Next Frontier for LLM Intelligence Expandingcited by 1
  32. 2024GPTA: Generative Prompt Tuning Assistant for Synergistic Downstream Neural Network Enhancement with LLMscited by 0
  33. 2024Prose-to-P4: Leveraging High Level Languagescited by 0
  34. 2024NSP: A Neuro-Symbolic Natural Language Navigational Plannercited by 0
  35. 2024PALLM: Evaluating and Enhancing PALLiative Care Conversations with Large Language Modelscited by 0
  36. 2025textttSEM-CTRL: Semantically Controlled Decodingcited by 0
  37. 2026Cluster Workload Allocation: Semantic Soft Affinity Using Natural Language Processingcited by 0
  38. 2026How Creative Are Large Language Models in Generating Molecules?cited by 0

50 most cited of 58 All 58 (JSON)