mlsanityData to 14 Oct 2026

RLVMR: Reinforcement Learning with Verifiable Meta-Reasoning Rewards for Robust Long-Horizon Agents

arXiv 2507.22844 ↗First posted 30 Jul 2025cs.LG cs.AIcited by 58

Not on the map. Why?

Cited by 41 in LLM agents and tool use, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

The development of autonomous agents for complex, long-horizon tasks is a central goal in AI. However, dominant training paradigms face a critical limitation: reinforcement learning (RL) methods that optimize solely for final task success often reinforce flawed or inefficient reasoning paths, a problem we term inefficient exploration. This leads to agents that are brittle and fail to generalize, as they learn to find solutions without learning how to reason coherently. To address this, we introduce RLVMR, a novel framework that integrates dense, process-level supervision into end-to-end RL by rewarding verifiable, meta-reasoning behaviors. RLVMR equips an agent to explicitly tag its cognitive steps, such as planning, exploration, and reflection, and provides programmatic, rule-based rewards for actions that contribute to effective problem-solving. These process-centric rewards are combined with the final outcome signal and optimized using a critic-free policy gradient method. On the challenging ALFWorld and ScienceWorld benchmarks, RLVMR achieves new state-of-the-art results, with our 7B model reaching an 83.6% success rate on the most difficult unseen task split. Our analysis confirms these gains stem from improved reasoning quality, including significant reductions in redundant actions and enhanced error recovery, leading to more robust, efficient, and interpretable agents.

Key facts. Cited by 58 papers in the map (data through 14 Oct 2026). First posted on arXiv 30 Jul 2025.

Builds on

0 of 0 references.

No references here yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 16; 2026: 42.

58 papers.

Sort:

By area:

  1. 2025The Landscape of Agentic Reinforcement Learning for LLMs: A Surveycited by 205
  2. 2026Hindsight Credit Assignment for Long-Horizon LLM Agentscited by 65
  3. 2026A Survey of Agentic Reasoning for Large Language Models: Towards Recursively Self-Improving and Collective Agentscited by 50
  4. 2025Agentic Reinforcement Learning with Implicit Step Rewardscited by 47
  5. 2025SALT: Step-level Advantage Assignment for Long-horizon Agents via Trajectory Graphcited by 28
  6. 2026EvoTool: Self-Evolving Tool-Use Policy Optimization in LLM Agents via Blame-Aware Mutation and Diversity-Aware Selectioncited by 24
  7. 2025DeepWideSearch: Benchmarking Depth and Width in Agentic Information Seekingcited by 21
  8. 2026Exploratory Memory-Augmented LLM Agent via Hybrid On- and Off-Policy Optimizationcited by 19
  9. 2026Milestone-Guided Policy Learning for Long-Horizon Language Agentscited by 19
  10. 2025EPO: Entropy-regularized Policy Optimization for LLM Agents Reinforcement Learningcited by 17
  11. 2026Exploring Reasoning Reward Model for Agentscited by 15
  12. 2025Stop-RAG: Value-Based Retrieval Control for Iterative RAGcited by 14
38 more
  1. 2026PivotRL: High Accuracy Agentic Post-Training at Low Compute Costcited by 14
  2. 2025Learn the Ropes, Then Trust the Wins: Self-imitation with Progressive Exploration for Agentic Reinforcement Learningcited by 11
  3. 2026Spark: Strategic Policy-Aware Exploration via Dynamic Branching for Long-Horizon Agentic Learningcited by 11
  4. 2026Search-R2: Enhancing Search-Integrated Reasoning via Actor-Refiner Collaborationcited by 11
  5. 2026Understanding Reasoning in LLMs through Strategic Information Allocation under Uncertaintycited by 9
  6. 2026WebAnchor: Anchoring Agent Planning to Stabilize Long-Horizon Web Reasoningcited by 8
  7. 2026RoboAgent: Chaining Basic Capabilities for Embodied Task Planningcited by 7
  8. 2026Recursive Self-Evolving Agents via Held-Out Selectioncited by 7
  9. 2026On Information Self-Locking in Reinforcement Learning for Active Reasoning of LLM agentscited by 6
  10. 2026TAPO: Translation Augmented Policy Optimization for Multilingual Mathematical Reasoningcited by 6
  11. 2025The Cognitive Bandwidth Bottleneck: Shifting Long-Horizon Agent from Planning with Actions to Planning with Schemascited by 5
  12. 2026Rethinking the Role of Entropy in Optimizing Tool-Use Behaviors for Large Language Model Agentscited by 5
  13. 2026Think Fast and Slow: Step-Level Cognitive Depth Adaptation for LLM Agentscited by 5
  14. 2026Towards trustworthy agentic AI: a comprehensive survey of safety, robustness, privacy, and system securitycited by 5
  15. 2026Bridging the Agent-World Gap: Text World Models for LLM-based Agentscited by 5
  16. 2025Reducing Belief Deviation in Reinforcement Learning for Active Reasoningcited by 4
  17. 2025Gradient Coupling: The Hidden Barrier to Generalization in Agentic Reinforcement Learningcited by 3
  18. 2025Verifying Meta-Awareness via Predictive Rewards in Reasoning Modelscited by 3
  19. 2026PRIME: Training Free Proactive Reasoning via Iterative Memory Evolution for User-Centric Agentcited by 3
  20. 2025OpenReward: Learning to Reward Long-form Agentic Tasks via Reinforcement Learningcited by 2
  21. 2025Thinking by Doing: Building Efficient World Model Reasoning in LLMs via Multi-turn Interactioncited by 2
  22. 2026MagicAgent: Towards Generalized Agent Planningcited by 2
  23. 2026TAPO: Transition-Aware Policy Optimization for LLM Agentscited by 2
  24. 2026State2State: Environment-Derived Mid-Training for LLM Agentscited by 2
  25. 2025RoboGPT-R1: Enhancing Robot Task Planning with Reinforcement Learningcited by 1
  26. 2025Training High-Level Schedulers with Execution-Feedback Reinforcement Learning for Long-Horizon GUI Automationcited by 1
  27. 2025Dual-Evolving Reinforcement Learning for Automated Reward Designcited by 1
  28. 2026Beyond Meta-Reasoning: Metacognitive Consolidation for Self-Improving LLM Reasoningcited by 1
  29. 2026DPEPO: Diverse Parallel Exploration Policy Optimization for LLM-based Agentscited by 1
  30. 2026Learning to Explore: Scaling Agentic Reasoning via Exploration-Aware Policy Optimizationcited by 1
  31. 2026Dynamic Mixture of Latent Memories for Self-Evolving Agentscited by 1
  32. 2026Maestro: Reinforcement Learning to Orchestrate Hierarchical Model-Skill Ensemblescited by 1
  33. 2026Policy-Conditioned Counterfactual Credit for Verifiable Reinforcement Learning of Long-Horizon Language Agentscited by 1
  34. 2026Semantic Consistency Policy Optimization for Reinforcement Learning of LLM Agentscited by 1
  35. 2026STAPO: Selective Trajectory-Aware Policy Optimization for LLM Agent Trainingcited by 1
  36. 2026Towards Better Agents for Multi-Turn User Interaction: The Next User Turn Is More Than Contextcited by 1
  37. 2026Agent-G2: Gaussian Guidance for Agentic Reinforcement Learningcited by 1
  38. 2026Diagnosing Task Insensitivity in Language Agentscited by 0

50 most cited of 58 All 58 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2026Verifiable Process Rewards for Agentic Reasoning
  2. 2026From RLVR to RLSVR: Task Transformation Induces Self-Verifiable Rewards for Open-Ended LLM Self-Improvement
  3. 2025RLPR: Extrapolating RLVR to General Domains without Verifiers
  4. 2025Reinforcement Learning Meets Large Language Models: A Survey of Advancements and Applications Across the LLM Lifecycle
  5. 2025Demystifying Reinforcement Learning in Agentic Reasoning
  6. 2025The Reasoning Boundary Paradox: How Reinforcement Learning Constrains Language Models
  7. 2026AgentV-RL: Scaling Reward Modeling with Agentic Verifier
  8. 2026Beyond Reward Engineering: A Data Recipe for Long-Context Reinforcement Learning