mlsanityData to 14 Oct 2026

Trading Human Curation for Synthetic Augmentation in RLVR

arXiv 2606.03800 ↗First posted 2 Jun 2026cs.LG cs.AIcited by 0

On the map: LLM agents and tool use → Reinforcement learning for LLM agents

Abstract (arXiv)

The supply of high-quality training tasks is a central bottleneck for reinforcement learning from verifiable rewards (RLVR) on agentic language models. Each task requires a sandboxed setup, a prompt, and a hand-authored reward function, and only tasks that pass a quality bar produce useful training signal. Hand-curation at this quality bar does not scale economically to the task counts effective RL training requires, and the substitution rate between automatically generated task variants and human-authored ones is not yet established. We investigate using pre-specified, gate-filtered augmentations of a small hand-authored base as a substitute for additional human curation during RLVR. We formalize the cost-adjusted trade rate ρcost between augmented and human-authored tasks, measure it through a controlled ablation across training corpora with varying augmentation share, and characterize the end-to-end economics of the augmentation pipeline. Substituting augmented content for additional human-authored tasks retains aggregate held-out generalization on a ten-benchmark suite spanning code, instruction following, reasoning, and multi-turn agentic function-calling. The cost-adjusted trade rate ρcost between gated synthetic and human-authored RLVR tasks stays in [1.4×, 11.6×] across the plausible chuman/caug range.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Reinforcement learning for LLM agents, one of 16 directions in LLM agents and tool use. Builds on 28 of its 31 references that are in the map. First posted on arXiv 2 Jun 2026.

Builds on

28 of 31 references.

Closest ancestors · 3

Same direction, newest first

  1. 2026ProRL Agent: Rollout-as-a-Service for RL Training of Multi-Turn LLM Agents19 Mar
  2. 2025AgentRL: Scaling Agentic Reinforcement Learning with a Multi-Turn, Multi-Task Framework5 Oct
  3. 2025Agent-RLVR: Training Software Engineering Agents via Guidance and Environment Rewards13 Jun

Other references · 25

  1. 2021Evaluating Large Language Models Trained on Codecited by 8,320
  2. 2023Let's Verify Step by Stepcited by 3,789
  3. 2023GPQA: A Graduate-Level Google-Proof Q&A Benchmarkcited by 2,879
  4. 2022Self-Instruct: Aligning Language Models with Self-Generated Instructionscited by 2,724
  5. 2024MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmarkcited by 1,848
  6. 2023Instruction-Following Evaluation for Large Language Modelscited by 1,228
  7. 2023WizardLM: Empowering large pre-trained language models to follow complex instructionscited by 1,125
  8. 2024Tulu 3: Pushing Frontiers in Open Language Model Post-Trainingcited by 1,053
  9. 2025Does Reinforcement Learning Really Incentivize Reasoning Capacity in LLMs Beyond the Base Model?cited by 982
  10. 2023OpenAssistant Conversations -- Democratizing Large Language Model Alignmentcited by 721
  11. 2025τ2-Bench: Evaluating Conversational Agents in a Dual-Control Environmentcited by 627
  12. 2020Image Augmentation Is All You Need: Regularizing Deep Reinforcement Learning from Pixelscited by 585
13 more
  1. 2020Reinforcement Learning with Augmented Datacited by 468
  2. 2022DS-1000: A Natural and Reliable Benchmark for Data Science Code Generationcited by 452
  3. 2024Hybrid LLM: Cost-Efficient and Quality-Aware Query Routingcited by 372
  4. 2024Training Software Engineering Agents and Verifiers with SWE-Gymcited by 336
  5. 2020Emergent Complexity and Zero-shot Transfer via Unsupervised Environment Designcited by 275
  6. 2024Token-Budget-Aware LLM Reasoningcited by 247
  7. 2020Prioritized Level Replaycited by 198
  8. 2025BIG-Bench Extra Hardcited by 125
  9. 2016The State of Applied Econometrics - Causality and Policy Evaluationcited by 41
  10. 2025Med-RLVR: Emerging Medical Reasoning from a 3B base model via reinforcement Learningcited by 40
  11. 2025TinyV: Reducing False Negatives in Verification Improves RL for LLM Reasoningcited by 21
  12. 2016Embracing Error to Enable Rapid Crowdsourcingcited by 15
  13. 2026ReSyn: Autonomously Scaling Synthetic Environments for Reasoning Modelscited by 5

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026From RLVR to RLSVR: Task Transformation Induces Self-Verifiable Rewards for Open-Ended LLM Self-Improvement
  2. 2025Position: The Hidden Costs and Measurement Gaps of Reinforcement Learning with Verifiable Rewards
  3. 2025Reinforcement Learning Meets Large Language Models: A Survey of Advancements and Applications Across the LLM Lifecycle
  4. 2026Reward Hacking Benchmark: Measuring Exploits in LLM Agents with Tool Use
  5. 2025Extending RLVR to Open-Ended Tasks via Verifiable Multiple-Choice Reformulation
  6. 2026Skill-R1: Agent Skill Evolution via Reinforcement Learning
  7. 2025Every Question Has Its Own Value: Reinforcement Learning with Explicit Human Values
  8. 2025Tool Zero: Training Tool-Augmented LLMs via Pure RL from Scratch