mlsanityData to 14 Oct 2026

RL-Struct: A Lightweight Reinforcement Learning Framework for Reliable Structured Output in LLMs

arXiv 2512.00319 ↗First posted 29 Nov 2025cs.AI cs.LGcited by 3

Not on the map. Why?

Abstract (arXiv)

The Structure Gap between probabilistic LLM generation and deterministic schema requirements hinders automated workflows. We propose RL-Struct, a lightweight framework using Gradient Regularized Policy Optimization (GRPO) with a hierarchical reward function to align LLMs with structural constraints. This approach eliminates the critic network, reducing peak VRAM by 38% compared to PPO. On complex JSON tasks, RL-Struct achieves 89.7% structural accuracy and 92.1% validity, significantly outperforming SFT and zero-shot baselines. We also report an emergent curriculum--a self-organized learning process where the model prioritizes syntax before semantics. Our model is publicly available at https://huggingface.co/Freakz3z/Qwen-JSON.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Builds on 75 of its 101 references that are in the map. First posted on arXiv 29 Nov 2025.

Builds on

75 of 101 references.

References · 75

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2020Language Models are Few-Shot Learnerscited by 32,419
  4. 2023GPT-4 Technical Reportcited by 18,396
  5. 2022Training language models to follow instructions with human feedbackcited by 15,971
  6. 2017Proximal Policy Optimization Algorithmscited by 15,215
  7. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  8. 2022Chain-of-Thought Prompting Elicits Reasoning in Large Language Modelscited by 12,459
  9. 2015Distilling the Knowledge in a Neural Networkcited by 10,324
  10. 2021Training Verifiers to Solve Math Word Problemscited by 10,109
  11. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  12. 2021Evaluating Large Language Models Trained on Codecited by 8,320
38 more
  1. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  2. 2022Self-Consistency Improves Chain of Thought Reasoning in Language Modelscited by 5,708
  3. 2022PaLM: Scaling Language Modeling with Pathwayscited by 5,326
  4. 2022Large Language Models are Zero-Shot Reasonerscited by 4,374
  5. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,276
  6. 2019Parameter-Efficient Transfer Learning for NLPcited by 4,236
  7. 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,915
  8. 2017Deep reinforcement learning from human preferencescited by 3,912
  9. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,795
  10. 2023Let's Verify Step by Stepcited by 3,789
  11. 2021Program Synthesis with Large Language Modelscited by 3,637
  12. 2023Generative Agents: Interactive Simulacra of Human Behaviorcited by 3,568
  13. 2022OPT: Open Pre-trained Transformer Language Modelscited by 3,512
  14. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,442
  15. 2023QLoRA: Efficient Finetuning of Quantized LLMscited by 3,418
  16. 2023Qwen Technical Reportcited by 3,344
  17. 2023Tree of Thoughts: Deliberate Problem Solving with Large Language Modelscited by 3,211
  18. 2023Mistral 7Bcited by 3,164
  19. 2015Trust Region Policy Optimizationcited by 2,580
  20. 2023A Survey on Large Language Model based Autonomous Agentscited by 2,242
  21. 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,191
  22. 2023Code Llama: Open Foundation Models for Codecited by 2,139
  23. 2022BLOOM: A 176B-Parameter Open-Access Multilingual Language Modelcited by 1,984
  24. 2023MetaGPT: Meta Programming for A Multi-Agent Collaborative Frameworkcited by 1,808
  25. 2022Fast Inference from Transformers via Speculative Decodingcited by 1,653
  26. 2022Competition-Level Code Generation with AlphaCodecited by 1,462
  27. 2023The Rise and Potential of Large Language Model Based Agents: A Surveycited by 1,439
  28. 2022Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?cited by 1,434
  29. 2022Least-to-Most Prompting Enables Complex Reasoning in Large Language Modelscited by 1,407
  30. 2023CAMEL: Communicative Agents for "Mind" Exploration of Large Language Model Societycited by 1,395
  31. 2023PaLM 2 Technical Reportcited by 1,162
  32. 2022Measuring and Narrowing the Compositionality Gap in Language Modelscited by 1,075
  33. 2023Large Language Models Cannot Self-Correct Reasoning Yetcited by 1,017
  34. 2023Teaching Large Language Models to Self-Debugcited by 988
  35. 2023Training Diffusion Models with Reinforcement Learningcited by 982
  36. 2023A General Theoretical Paradigm to Understand Learning from Human Preferencescited by 912
  37. 2022Solving math word problems with process- and outcome-based feedbackcited by 901
  38. 2022CodeGen: An Open Large Language Model for Code with Multi-Turn Program Synthesiscited by 885

50 most cited of 75 All 75 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 3.

3 papers.

  1. 2026GenCircuit-RL: Reinforcement Learning from Hierarchical Verification for Genetic Circuit Designcited by 0
  2. 2026Improving LLM-Generated Process Model Quality Through Reinforcement Learning: The Role of Reward Function Designcited by 0
  3. 2026Structural Chain-of-Thought: Reasoning with Executable Computational Graphscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Token-Efficient RL for LLM Reasoning
  2. 2025A Technical Survey of Reinforcement Learning Techniques for Large Language Models
  3. 2026STO-RL: Offline RL under Sparse Rewards via LLM-Guided Subgoal Temporal Order
  4. 2026Right Values, Wrong Places: Diagnosing and Fixing Record Misplacement in LLM-Generated JSON
  5. 2026PlexRL: Cluster-Level Orchestration of Serviceized LLM Execution for RLVR
  6. 2026GradAlign: Gradient-Aligned Data Selection for LLM Reinforcement Learning
  7. 2025From Data-Centric to Sample-Centric: Enhancing LLM Reasoning via Progressive Optimization
  8. 2025AsyncFlow: An Asynchronous Streaming RL Framework for Efficient LLM Post-Training