mlsanityData to 14 Oct 2026

ARS: Agentic Reward System for Robot Learning

arXiv 2609.34484 ↗First posted 28 Sep 2026cs.RO cs.LGcited by 0

On the map: LLM agents and tool use → Reinforcement learning for LLM agents

also in 1 more
Abstract (arXiv)

Progress reward modeling is the problem of estimating how a robot's behavior changes task progress over time. Reliable estimation requires distinguishing meaningful state changes from failed attempts and task-irrelevant actions. We introduce the Agentic Reward System (ARS), an inference framework for progress reward modeling with general-purpose vision-language models (VLMs), without additional reward-model training. Given an offline trajectory and a task instruction, ARS uses adaptive visual inspection for both event proposal and verification. A subagent proposes a task-relevant event timeline, which a primary agent verifies and revises before estimating per-frame progress. ARS can incorporate optional terminal outcome labels and visual references to inform its judgments. It can also audit progress estimates from external reward models. We evaluate ARS with a 27B VLM on a controlled semantic-mismatch benchmark and downstream policy learning in simulation and on a real robot. The benchmark reveals that several evaluated reward baselines assign spurious progress to wrong-object manipulation even in simple pick-and-place scenes. ARS better suppresses these errors and outperforms these baselines in simulation policy learning. We further demonstrate that ARS supports long-horizon policy learning from mixed-quality offline experience on real-robot multi-screw fastening in a full-scale laboratory replica of an industrial washing-machine assembly line. These results suggest that structured inference and verification can improve the usefulness of general-purpose VLMs for robot reward modeling. Code is at https://github.com/midea-ai/ars

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Reinforcement learning for LLM agents, one of 16 directions in LLM agents and tool use. Builds on 45 of its 47 references that are in the map. First posted on arXiv 28 Sep 2026.

Builds on

45 of 47 references.

Closest ancestors · 1

Same direction, newest first

  1. 2026AgentV-RL: Scaling Reward Modeling with Agentic Verifier17 Apr

Other references · 44

  1. 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,695
  2. 2025Qwen3-VL Technical Reportcited by 2,793
  3. 2025π0.5: a Vision-Language-Action Model with Open-World Generalizationcited by 1,761
  4. 2023LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learningcited by 1,410
  5. 2025Fine-Tuning Vision-Language-Action Models: Optimizing Speed and Successcited by 883
  6. 2021What Matters in Learning from Offline Human Demonstrations for Robot Manipulationcited by 725
  7. 2022VIP: Towards Universal Visual Reward and Representation via Value-Implicit Pre-Trainingcited by 395
  8. 2024VideoAgent: Long-form Video Understanding with Large Language Model as Agentcited by 332
  9. 2025π*0.6: a VLA That Learns From Experiencecited by 303
  10. 2023LIV: Language-Image Representations and Rewards for Robotic Controlcited by 188
  11. 2024RL-VLM-F: Reinforcement Learning from Vision Language Foundation Model Feedbackcited by 156
  12. 2023RoboCLIP: One Demonstration is Enough to Learn Robot Policiescited by 134
32 more
  1. 2025What Can RL Bring to VLA Generalization? An Empirical Studycited by 108
  2. 2024Vision Language Models are In-Context Value Learnerscited by 90
  3. 2025ReWiND: Language-Guided Rewards Teach Robot Policies without New Demonstrationscited by 60
  4. 2026Robometer: Scaling General-Purpose Robotic Reward Models via Trajectory Comparisonscited by 60
  5. 2026RoboReward: General-Purpose Vision-Language Reward Models for Roboticscited by 53
  6. 2025SARM: Stage-Aware Reward Modeling for Long Horizon Robot Manipulationcited by 42
  7. 2026TOPReward: Token Probabilities as Hidden Zero-Shot Rewards for Roboticscited by 33
  8. 2024Rank2Reward: Learning Shaped Reward Functions from Passive Videocited by 32
  9. 2025RoVer: Robot Reward Model as Test-Time Verifier for Vision-Language-Action Modelcited by 19
  10. 2026PROGRESSLM: Towards Progress Reasoning in Vision-Language Modelscited by 15
  11. 2024PROGRESSOR: A Perceptually Guided Reward Estimator with Self-Supervised Online Refinementcited by 13
  12. 2025ARM-Thinker: Reinforcing Multimodal Generative Reward Models with Agentic Tool Use and Visual Reasoningcited by 13
  13. 2024VICtoR: Learning Hierarchical Vision-Instruction Correlation Rewards for Long-horizon Manipulationcited by 12
  14. 2025ProRe: A Proactive Reward System for GUI Agents via Reasoner-Actor Collaborationcited by 11
  15. 2026Large Reward Models: Generalizable Online Robot Reward Generation with Vision-Language Modelscited by 11
  16. 2026ARM: Advantage Reward Modeling for Long-Horizon Manipulationcited by 10
  17. 2026PRM-as-a-Judge: A Dense Evaluation Paradigm for Fine-Grained Robotic Auditingcited by 9
  18. 2026ProcVLM: Learning Procedure-Grounded Progress Rewards for Robotic Manipulationcited by 9
  19. 2025ROVER: Recursive Reasoning Over Videos with Vision-Language Models for Embodied Taskscited by 6
  20. 2025TimeRewarder: Learning Dense Reward from Passive Videos via Frame-wise Temporal Distancecited by 6
  21. 2026Recurrent Reasoning with Vision-Language Models for Estimating Long-Horizon Embodied Task Progresscited by 6
  22. 2026TapSampling: Inference-Time Sampling with a Task-Progress-Understanding Verifier for Robotic Manipulationcited by 5
  23. 2026Agentic Reward Modeling: Verifying GUI Agent via Progressive Trajectory-Grounded Interactioncited by 4
  24. 2026SOLE-R1: Video-Language Reasoning as the Sole Reward for On-Robot Reinforcement Learningcited by 3
  25. 2026EV-WM: Event-Verified World Models for Long-Horizon Robotic Manipulationcited by 3
  26. 2026Reward as An Agent for Embodied World Modelscited by 3
  27. 2026RARM: Confidence-Gated Progress Reward Modeling for RL in Manipulationcited by 3
  28. 2026DenseReward: Dense Reward Learning via Failure Synthesis for Robotic Manipulationcited by 2
  29. 2026From Demonstrations to Rewards: Test-Time Prompt Optimization for VLM Reward Modelscited by 1
  30. 2026WARP-RM: A Warp-Augmented Relative Progress Reward Model for Data Curationcited by 1
  31. 2026Stage-Transition Dense Reward Modeling for Reinforcement Learningcited by 1
  32. 2026UR-VC: Unsupervised Robotic Value Correction for Time-Derived Progress Proxiescited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Intrinsic Robot Rewarding: Reusing VLA Representations for Autonomous Evaluation and Policy Improvement
  2. 2026SARM2: Multi-Task Stage Aware Reward Modeling for Self Improving Robotic Manipulation
  3. 2025VARP: Reinforcement Learning from Vision-Language Model Feedback with Agent Regularized Preferences
  4. 2026Progress Reward Modeling for Robotic Learning: A Comprehensive Survey
  5. 2025Training-free Generation of Temporally Consistent Rewards from VLMs
  6. 2025A Vision-Language-Action-Critic Model for Robotic Real-World Reinforcement Learning
  7. 2026Find Something You Can't Do: Agentic Real-World Reinforcement Learning for Self-Improving VLA Models
  8. 2026Reward-DAgger: Robot-Gated Interactive Imitation Learning with General-Purpose Progress-Based Reward Models