mlsanityData to 7 Oct 2026

ContractRL: Shielded Group-Relative Policy Optimization for Auditable Tool-Call Repair

arXiv 2610.00328 ↗First posted 29 Sep 2026cs.AI cs.CL cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Structured tool calls often fail after only a small number of fields violate a schema or an execution contract. Regenerating the complete object enlarges the action surface and makes repeated repair difficult to audit. We introduce ContractRL, a contract-constrained sequential repair protocol that models verifier-guided JSON repair as a bounded decision process. At each step the policy observes the candidate, typed verifier feedback, JSON Pointer, immutable repair history, and remaining budget; a contract-derived action mask filters malformed or prohibited RFC-6902 operations before a deterministic validator performs the transition. We specify a contract-constrained group-relative objective for patch, retry, and abstention decisions while keeping canonical targets and semantic labels outside the online state until trace freeze. Under identical verifier information, ContractRL attains 0.9362 semantic success with 34.4 generated tokens, compared with 0.9076 and 44.9 tokens for Patch-SFT and 0.9148 and 137.2 tokens for full regeneration over 192 cases per seed and five seeds. Policy optimization improves semantic success from 0.9186 for supervised ContractRL to 0.9375. A separate three-seed paired evaluation against Patch-SFT yields a semantic difference of +0.0396 (95% CI [+0.0137,+0.0662], p=0.0039). Feedback, action-mask, budget, and schema-shift analyses connect these gains to localized correction, while adversarial and multi-turn evaluations characterize the remaining failure modes.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 35 of its 39 references that are in the map. First posted on arXiv 29 Sep 2026.

Builds on

35 of 39 references.

References · 35

  1. 2022Training language models to follow instructions with human feedbackcited by 15,712
  2. 2017Proximal Policy Optimization Algorithmscited by 14,441
  3. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  4. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  5. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
  6. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
  7. 2023Reflexion: Language Agents with Verbal Reinforcement Learningcited by 4,064
  8. 2023Let's Verify Step by Stepcited by 3,598
  9. 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,317
  10. 2023SWE-bench: Can Language Models Resolve Real-World GitHub Issues?cited by 3,193
  11. 2023Voyager: An Open-Ended Embodied Agent with Large Language Modelscited by 2,081
  12. 2023WebArena: A Realistic Web Environment for Building Autonomous Agentscited by 1,908
23 more
  1. 2020Conservative Q-Learning for Offline Reinforcement Learningcited by 1,777
  2. 2024SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineeringcited by 1,628
  3. 2021Decision Transformer: Reinforcement Learning via Sequence Modelingcited by 1,528
  4. 2020D4RL: Datasets for Deep Data-Driven Reinforcement Learningcited by 1,406
  5. 2024τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domainscited by 1,280
  6. 2021Offline Reinforcement Learning with Implicit Q-Learningcited by 1,245
  7. 2023AgentBench: Evaluating LLMs as Agentscited by 1,211
  8. 2024OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environmentscited by 1,134
  9. 2022Planning with Diffusion for Flexible Behavior Synthesiscited by 1,023
  10. 2023Math-Shepherd: Verify and Reinforce LLMs Step-by-step without Human Annotationscited by 895
  11. 2023FrugalGPT: How to Use Large Language Models While Reducing Cost and Improving Performancecited by 791
  12. 2024RouteLLM: Learning to Route LLMs with Preference Datacited by 633
  13. 2024RewardBench: Evaluating Reward Models for Language Modelingcited by 480
  14. 2019SelectiveNet: A Deep Neural Network with an Integrated Reject Optioncited by 333
  15. 2024ProcessBench: Identifying Process Errors in Mathematical Reasoningcited by 277
  16. 2024ToolSandbox: A Stateful, Conversational, Interactive Evaluation Benchmark for LLM Tool Use Capabilitiescited by 272
  17. 2022Defining and Characterizing Reward Hackingcited by 184
  18. 2024XGrammar: Flexible and Efficient Structured Generation Engine for Large Language Modelscited by 73
  19. 2025JSONSchemaBench: A Rigorous Benchmark of Structured Outputs for Language Modelscited by 66
  20. 2025Learning to Generate Structured Output with Schema Reinforcement Learningcited by 27
  21. 2025SLOT: Structuring the Output of Large Language Modelscited by 18
  22. 2026PatchBoard: Schema-Grounded State Mutation for Reliable and Auditable LLM Multi-Agent Collaborationcited by 2
  23. 2026Structured Feedback Improves Repair in an LLM Agent Loopcited by 2

Built on by

Not cited yet.