Beyond Next-Token Prediction: An RLVR Proof of Concept for Tool-Use Agents on Atlassian Workflows
On the map: LLM agents and tool use → Reinforcement learning for LLM agents
Abstract (arXiv)
Large language models are trained to predict the next token, not to act inside a specific API. In niche enterprise SaaS workflows -- where success means hitting the right endpoint with the right nested arguments in the right order -- this objective mismatch shows up as silent failures: dropped required fields, hallucinated tools, or early stops after a single read. We ask whether Reinforcement Learning with Verifiable Rewards (RLVR), applied directly in the target environment, closes the gap. As a proof of concept we build a suite of five synthetic environments emulating the Jira REST v3 and Confluence v2 APIs at schema fidelity; rewards are computed entirely from the tool-call trace, with no live API, no learned judge, and no human label in the loop. Scoring prompted Qwen3-1.7B and Qwen3.5-4B on the same checkers that drive GRPO training, we find that on the four scenarios whose rewards are non-degenerate the RL-trained policy lifts average reward from a 4B-baseline range of 0.35--0.92 to 0.95--1.00, with the largest single gain on Confluence page creation (0.35 rightarrow 1.00). We position this as a preliminary step toward outcome-optimised small models for niche enterprise APIs, and foreground two limitations a workshop reader should weigh: hand-crafting verifiable rewards does not scale beyond the handful of endpoints reported here, and one of our five scenarios (ticket-transition) has a saturating reward shape that the prompted 4B already maxes out.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Reinforcement learning for LLM agents, one of 16 directions in LLM agents and tool use. Builds on 6 of its 19 references that are in the map. First posted on arXiv 1 Jul 2026.
Builds on
6 of 19 references.
Closest ancestors · 1
Same direction, newest first
Other references · 5
- 2021Training Verifiers to Solve Math Word Problemscited by 9,743
- 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
- 2025Qwen3 Technical Reportcited by 8,195
- 2023Toolformer: Language Models Can Teach Themselves to Use Toolscited by 4,072
- 2023ToolLLM: Facilitating Large Language Models to Master 16000+ Real-world APIscited by 1,880
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026ToolRLA: Multiplicative Reward Decomposition for Tool-Integrated Agents
- 2026Synthesize and Reward -- Reinforcement Learning for Multi-Step Tool Use in Live Environments
- 2026Salesforce Koa: An Enterprise Language Model for Agentic Tool Use
- 2026Controllable and Verifiable Tool-Use Data Synthesis for Agentic Reinforcement Learning
- 2026Reward Hacking Benchmark: Measuring Exploits in LLM Agents with Tool Use
- 2026Contract2Tool: Learning Preconditions and Effects for Reliable Tool-Augmented LLM Agents
- 2026When Simulation Lies: A Sim-to-Real Benchmark and Domain-Randomized RL Recipe for Tool-Use Agents
- 2026Tool-R0: Self-Evolving LLM Agents for Tool-Learning from Zero Data