mlsanityData to 14 Oct 2026

IterGen: Iterative Semantic-aware Structured LLM Generation with Backtracking

arXiv 2410.07295 ↗First posted 9 Oct 2024cs.SE cs.LG cs.PLcited by 29

Not on the map. Why?

Abstract (arXiv)

Large Language Models (LLMs) are widely used for tasks such as natural language and code generation, but their outputs often suffer from issues like hallucination, toxicity, and incorrect results. Current libraries for structured LLM generation rely on left-to-right decoding without support for backtracking, limiting the ability to correct or refine outputs mid-generation. To address this, we introduce IterGen, a user-friendly library for iterative, grammar-guided LLM generation that enables users to move both forward and backward within the generated output based on grammar symbols. By leveraging a symbol-to-position mapping and maintaining the key-value (KV) cache state, IterGen ensures efficient and structured generation while allowing for corrections during the process. We demonstrate IterGen's effectiveness in two important applications: reducing privacy leakage in LLM outputs and improving the accuracy of LLM-generated SQL and Vega-Lite queries. Our code and additional resources are available at https://structuredllm.com.

Key facts. Cited by 29 papers in the map (data through 14 Oct 2026). Builds on 14 of its 24 references that are in the map. First posted on arXiv 9 Oct 2024.

Builds on

14 of 24 references.

References · 14

  1. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  2. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  3. 2018Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Taskcited by 1,139
  4. 2023DecodingTrust: A Comprehensive Assessment of Trustworthiness in GPT Modelscited by 518
  5. 2021PICARD: Parsing Incrementally for Constrained Auto-Regressive Decoding from Language Modelscited by 411
  6. 2024Hallucination is Inevitable: An Innate Limitation of Large Language Modelscited by 332
  7. 2023Grammar-Constrained Decoding for Structured NLP Tasks without Finetuningcited by 197
  8. 2022Synchromesh: Reliable code generation from pre-trained language modelscited by 156
  9. 2022Prompting Is Programming: A Query Language for Large Language Modelscited by 110
  10. 2023Copiloting the Copilots: Fusing Large Language Models with Completion Engines for Automated Program Repaircited by 55
  11. 2021Collecting and Characterizing Natural Language Utterances for Specifying Data Visualizationscited by 31
  12. 2023Exploring Parameter-Efficient Fine-Tuning Techniques for Code Generation with Large Language Modelscited by 28
2 more
  1. 2023Guiding Language Models of Code with Global Context using Monitorscited by 20
  2. 2024A Comparative Study of DSL Code Generation: Fine-Tuning vs. Optimized Retrieval Augmentationcited by 10

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 12; 2026: 17.

29 papers.

Sort:

By area:

  1. 2025CRANE: Reasoning with constrained LLM generationcited by 38
  2. 2025Enforcing Temporal Constraints for LLM Agentscited by 34
  3. 2025Generate, but Verify: Reducing Hallucination in Vision-Language Models with Retrospective Resamplingcited by 33
  4. 2025A Survey on Parallel Text Generation: From Parallel Decoding to Diffusion Language Modelscited by 30
  5. 2025CodeARC: Benchmarking Reasoning Capabilities of LLM Agents for Inductive Program Synthesiscited by 16
  6. 2026SEVerA: Verified Synthesis of Self-Evolving Agentscited by 8
  7. 2026Lookahead-then-Verify: Reliable Constrained Decoding for Diffusion LLMs under Context-Free Grammarscited by 7
  8. 2026The Hidden Cost of Structured Generation in LLMs: Draft-Conditioned Constrained Decodingcited by 7
  9. 2025TreeCoder: Systematic Exploration and Optimisation of Decoding and Constraints for LLM Code Generationcited by 5
  10. 2025Constrained Adaptive Rejection Samplingcited by 3
  11. 2025UTF-8 Plumbing: Byte-level Tokenizers Unavoidably Enable LLMs to Generate Ill-formed UTF-8cited by 3
  12. 2026The Format Taxcited by 3
17 more
  1. 2026Semantic Layers for Reliable LLM-Powered Data Analytics: A Paired Benchmark of Accuracy and Hallucination Across Three Frontier Modelscited by 3
  2. 2025BEAVER: An Efficient Deterministic LLM Verifiercited by 2
  3. 2026Token-Guard: Towards Token-Level Hallucination Control via Self-Checking Decodingcited by 2
  4. 2026VegaChat: A Robust Framework for LLM-Based Chart Generation and Assessmentcited by 1
  5. 2026TruncProof: A Guardrail for LLM-based JSON Generation under Token-Length Constraintscited by 1
  6. 2025textttSEM-CTRL: Semantically Controlled Decodingcited by 0
  7. 2025FASTGEN: Fast and Cost-Effective Synthetic Tabular Data Generation with LLMscited by 0
  8. 2025Componentization: Decomposing Monolithic LLM Responses into Manipulable Semantic Unitscited by 0
  9. 2026Story2Proposal: A Scaffold for Structured Scientific Paper Writingcited by 0
  10. 2026Scientific Knowledge-driven Decoding Constraints Improving the Reliability of LLMscited by 0
  11. 2026Generative Compilation: On-the-Fly Compiler Feedback as AI Generates Codecited by 0
  12. 2026Decode-Time Grammars: Constrained LLM Generation over a Refinement Order of Grammar Fragmentscited by 0
  13. 2026Imprompt: A Language Framework for Prompt Programmingcited by 0
  14. 2026Visualizing Graph-to-Answer Mechanism Recovery in Materials-Science Hypothesis Generationcited by 0
  15. 2026Parser Transitions Already Reveal Structural Responsibility: KV Persistence for Structured Generationcited by 0
  16. 2026Semantic Prefix Oracles for LLM Decoding: Contracts and Differential Validationcited by 0
  17. 2026Structural Automata for Neural Semantic Parsingcited by 0