mlsanityData to 7 Oct 2026

Beyond Next-Token Prediction: An RLVR Proof of Concept for Tool-Use Agents on Atlassian Workflows

arXiv 2607.01465 ↗First posted 1 Jul 2026cs.AIcited by 0

On the map: LLM agents and tool use → Reinforcement learning for LLM agents

Abstract (arXiv)

Large language models are trained to predict the next token, not to act inside a specific API. In niche enterprise SaaS workflows -- where success means hitting the right endpoint with the right nested arguments in the right order -- this objective mismatch shows up as silent failures: dropped required fields, hallucinated tools, or early stops after a single read. We ask whether Reinforcement Learning with Verifiable Rewards (RLVR), applied directly in the target environment, closes the gap. As a proof of concept we build a suite of five synthetic environments emulating the Jira REST v3 and Confluence v2 APIs at schema fidelity; rewards are computed entirely from the tool-call trace, with no live API, no learned judge, and no human label in the loop. Scoring prompted Qwen3-1.7B and Qwen3.5-4B on the same checkers that drive GRPO training, we find that on the four scenarios whose rewards are non-degenerate the RL-trained policy lifts average reward from a 4B-baseline range of 0.35--0.92 to 0.95--1.00, with the largest single gain on Confluence page creation (0.35 rightarrow 1.00). We position this as a preliminary step toward outcome-optimised small models for niche enterprise APIs, and foreground two limitations a workshop reader should weigh: hand-crafting verifiable rewards does not scale beyond the handful of endpoints reported here, and one of our five scenarios (ticket-transition) has a saturating reward shape that the prompted 4B already maxes out.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Reinforcement learning for LLM agents, one of 16 directions in LLM agents and tool use. Builds on 6 of its 19 references that are in the map. First posted on arXiv 1 Jul 2026.

Builds on

6 of 19 references.

Closest ancestors · 1

Same direction, newest first

  1. 2025Agent-RLVR: Training Software Engineering Agents via Guidance and Environment Rewards13 Jun

Other references · 5

  1. 2021Training Verifiers to Solve Math Word Problemscited by 9,743
  2. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
  3. 2025Qwen3 Technical Reportcited by 8,195
  4. 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
  5. 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIscited by 1,880

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026ToolRLA: Multiplicative Reward Decomposition for Tool-Integrated Agents
  2. 2026Synthesize and Reward -- Reinforcement Learning for Multi-Step Tool Use in Live Environments
  3. 2026Salesforce Koa: An Enterprise Language Model for Agentic Tool Use
  4. 2026Controllable and Verifiable Tool-Use Data Synthesis for Agentic Reinforcement Learning
  5. 2026Reward Hacking Benchmark: Measuring Exploits in LLM Agents with Tool Use
  6. 2026Contract2Tool: Learning Preconditions and Effects for Reliable Tool-Augmented LLM Agents
  7. 2026When Simulation Lies: A Sim-to-Real Benchmark and Domain-Randomized RL Recipe for Tool-Use Agents
  8. 2026Tool-R0: Self-Evolving LLM Agents for Tool-Learning from Zero Data