mlsanityData to 7 Oct 2026

Learning to Generate Structured Output with Schema Reinforcement Learning

arXiv 2502.18878 ↗First posted 26 Feb 2025cs.CLcited by 27

Not on the map. Why?

Abstract (arXiv)

This study investigates the structured generation capabilities of large language models (LLMs), focusing on producing valid JSON outputs against a given schema. Despite the widespread use of JSON in integrating language models with programs, there is a lack of comprehensive analysis and benchmarking of these capabilities. We explore various aspects of JSON generation, such as structure understanding, escaping, and natural language description, to determine how to assess and enable LLMs to generate valid responses. Building upon this, we propose SchemaBench features around 40K different JSON schemas to obtain and assess models' abilities in generating valid JSON. We find that the latest LLMs are still struggling to generate a valid JSON string. Moreover, we demonstrate that incorporating reinforcement learning with a Fine-grained Schema Validator can further enhance models' understanding of JSON schema, leading to improved performance. Our models demonstrate significant improvement in both generating JSON outputs and downstream tasks.

Key facts. Cited by 27 papers in the map (data through 7 Oct 2026). Builds on 27 of its 37 references that are in the map. First posted on arXiv 26 Feb 2025.

Builds on

27 of 37 references.

References · 27

  1. 2017Proximal Policy Optimization Algorithmscited by 14,441
  2. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,302
  3. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  4. 2020Measuring Massive Multitask Language Understandingcited by 7,768
  5. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,304
  6. 2021Measuring Mathematical Problem Solving With the MATH Datasetcited by 5,203
  7. 2018Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challengecited by 4,710
  8. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
  9. 2024Qwen2 Technical Reportcited by 2,076
  10. 2023Instruction-Following Evaluation for Large Language Modelscited by 1,143
  11. 2022GLM-130B: An Open Bilingual Pre-trained Modelcited by 837
  12. 2023Enhancing Chat Language Models by Scaling High-quality Instructional Conversationscited by 825
15 more
  1. 2024MiniCPM: Unveiling the Potential of Small Language Models with Scalable Training Strategiescited by 694
  2. 2023Tool Learning with Foundation Modelscited by 302
  3. 2023Using an LLM to Help With Code Understandingcited by 222
  4. 2024Advancing LLM Reasoning Generalists with Preference Treescited by 191
  5. 2023Grammar-Constrained Decoding for Structured NLP Tasks without Finetuningcited by 186
  6. 2022Synchromesh: Reliable code generation from pre-trained language modelscited by 156
  7. 2024Does Prompt Formatting Have Any Impact on LLM Performance?cited by 132
  8. 2023WebCPM: Interactive Web Search for Chinese Long-form Question Answeringcited by 93
  9. 2024Let Me Speak Freely? A Study on the Impact of Format Restrictions on Performance of Large Language Modelscited by 90
  10. 2024FOFO: A Benchmark to Evaluate LLMs' Format-Following Capabilitycited by 79
  11. 2025JSONSchemaBench: A Rigorous Benchmark of Structured Outputs for Language Modelscited by 66
  12. 2023Toolink: Linking Toolkit Creation and Using through Chain-of-Solving on Open-Source Modelcited by 29
  13. 2024Benchmarking Large Language Models on Controllable Generation under Diversified Instructionscited by 26
  14. 2025Verifiable Format Control for Large Language Model Generationscited by 16
  15. 2025LLMER: Crafting Interactive Extended Reality Worlds with JSON Data Generated by Large Language Modelscited by 7

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 9; 2026: 18.

27 papers.

Sort:

By area:

  1. 2025A Technical Survey of Reinforcement Learning Techniques for Large Language Modelscited by 19
  2. 2025SLOT: Structuring the Output of Large Language Modelscited by 18
  3. 2026Omni-RRM: Advancing Omni Reward Modeling via Automatic Rubric-Grounded Preference Synthesiscited by 7
  4. 2025PARSE: LLM Driven Schema Optimization for Reliable Entity Extractioncited by 6
  5. 2026A Survey on Evaluating Quality and Trustworthiness in LLM-Generated Datacited by 5
  6. 2026LLMStructBench: Benchmarking Large Language Model Structured Data Extractioncited by 4
  7. 2025When Grammar Guides the Attack: Uncovering Control-Plane Vulnerabilities in LLMs with Structured Outputcited by 3
  8. 2025Training-Free Multimodal Large Language Model Orchestrationcited by 2
  9. 2026The Extrapolation Cliff in On-Policy Distillation of Near-Deterministic Structured Outputscited by 2
  10. 2026Large Language Models for Agentic NetOps and AIOps: Architectures, Evaluation, and Safetycited by 2
  11. 2025Introducing HALC: A general pipeline for the systematic and reliable construction of prompts for automated coding with LLMs in the computational social sciencescited by 1
  12. 2026ADEMA: A Knowledge-State Orchestration Architecture for Long-Horizon Knowledge Synthesis with LLMAgentscited by 1
15 more
  1. 2026CuraView: A Multi-Agent Framework for Medical Hallucination Detection with GraphRAG-Enhanced Knowledge Verificationcited by 1
  2. 2025MeXtract: Light-Weight Metadata Extraction from Scientific Paperscited by 0
  3. 2025Structured Document Translation via Format Reinforcement Learningcited by 0
  4. 2025CienaLLM: Generative Climate-Impact Extraction from News Articles with Autoregressive LLMscited by 0
  5. 2026ALGOGEN: Tool-Generated Verifiable Traces for Reliable Algorithm Visualizationcited by 0
  6. 2026SPIN: Structural LLM Planning via Iterative Navigation for Industrial Taskscited by 0
  7. 2026Ontology-constrained multi-LLM scoring of hypothesis support in the predictive processing literaturecited by 0
  8. 2026Source-Grounded Data Generation for Text-to-JSON Learningcited by 0
  9. 2026ASMR: Agentic Schema Generation for Ship Maintenance Report Writingcited by 0
  10. 2026QuantumMind: Constraint-Grounded Agentic Reasoning for Speedup Analysis in Quantum Computingcited by 0
  11. 2026The Parser Already Knows: Lightweight Bias Correction in Constrained Decodingcited by 0
  12. 2026TRACE-BN: Transferring Bangla-English Tutoring Behavior to a Sub-1B Offline Language Modelcited by 0
  13. 2026Grounded Normative Rule Generation with Structured Searchcited by 0
  14. 2026GrammarRL: Effective Grammar-Constrained Decoding via Reinforcement Learningcited by 0
  15. 2026ContractRL: Shielded Group-Relative Policy Optimization for Auditable Tool-Call Repaircited by 0