mlsanityData to 13 Oct 2026

Progress Reward Modeling for Robotic Learning: A Comprehensive Survey

arXiv 2607.21655 ↗First posted 22 Jul 2026cs.RO cs.CLcited by 5

On the map: LLM alignment / RLHF / preference optimisation → Reward modeling and RLHF

Abstract (arXiv)

Robotic learning takes place in dynamic environments with large behavior spaces. A terminal success signal only tells the robot whether the task is completed. It does not explain whether the current behavior is making progress, remaining unchanged, or undoing earlier progress. For this reason, recent studies have increasingly explored progress rewards that provide feedback during task execution. However, the current literature lacks a shared framework. Existing methods use different observations, goal specifications, output signals, supervision sources, and evaluation protocols. This makes it difficult to compare them and understand what their results actually validate. In this survey, we provide a unified view of progress reward modeling for robotic learning. We organize the field in three connected steps. We first study the interface of a progress model. This defines the problem from the outside by asking what information the model receives and what form of progress signal it produces. We then move inside the model and study the methods used to construct this signal. This reveals the different assumptions and mechanisms behind progress estimation and reward generation. Finally, we examine the data and benchmarks that support these methods. This shows how progress supervision is obtained and what different evaluations actually measure. Together, these three perspectives connect what a progress model is, how it is built, and how its quality is validated. We further summarize the main limitations of current approaches and discuss future research directions.

Key facts. Cited by 5 papers in the map (data through 13 Oct 2026). Placed in Reward modeling and RLHF, one of 10 directions in LLM alignment / RLHF / preference optimisation. Builds on 43 of its 54 references that are in the map. First posted on arXiv 22 Jul 2026.

Builds on

43 of 54 references.

Closest ancestors · 7

Same direction, newest first

  1. 2026Robometer: Scaling General-Purpose Robotic Reward Models via Trajectory Comparisons2 Mar
  2. 2026TOPReward: Token Probabilities as Hidden Zero-Shot Rewards for Robotics22 Feb
  3. 2024Video-Language Critic: Transferable Reward Functions for Language-Conditioned Robotics30 May
  4. 2023Eureka: Human-Level Reward Design via Coding Large Language Models19 Oct
  5. 2023Vision-Language Models are Zero-Shot Reward Models for Reinforcement Learning19 Oct
  6. 2023Text2Reward: Reward Shaping with Language Models for Reinforcement Learning20 Sep
  7. 2023PEARL: Zero-shot Cross-task Preference Alignment and Robust Reward Learning for Robotic Manipulation6 Jun

Other references · 36

  1. 2025π0.5: a Vision-Language-Action Model with Open-World Generalizationcited by 1,699
  2. 2025GR00T N1: An Open Foundation Model for Generalist Humanoid Robotscited by 1,197
  3. 2022R3M: A Universal Visual Representation for Robot Manipulationcited by 683
  4. 2026World Action Models are Zero-shot Policiescited by 438
  5. 2022VIP: Towards Universal Visual Reward and Representation via Value-Implicit Pre-Trainingcited by 385
  6. 2023Language to Rewards for Robotic Skill Synthesiscited by 309
  7. 2023LIV: Language-Image Representations and Rewards for Robotic Controlcited by 181
  8. 2024RL-VLM-F: Reinforcement Learning from Vision Language Foundation Model Feedbackcited by 148
  9. 2023Vision-Language Models as Success Detectorscited by 140
  10. 2026π0.7: a Steerable Generalist Robotic Foundation Model with Emergent Capabilitiescited by 131
  11. 2021Learning Language-Conditioned Robot Behavior from Offline Data and Crowd-Sourced Annotationcited by 118
  12. 2019End-to-End Robotic Reinforcement Learning without Reward Engineeringcited by 116
24 more
  1. 2021Learning Generalizable Robotic Reward Functions from "In-The-Wild" Human Videoscited by 116
  2. 2024Vision Language Models are In-Context Value Learnerscited by 84
  3. 2023Video Prediction Models as Rewards for Reinforcement Learningcited by 76
  4. 2022Can Foundation Models Perform Zero-Shot Task Specification For Robot Manipulation?cited by 70
  5. 2025A Vision-Language-Action-Critic Model for Robotic Real-World Reinforcement Learningcited by 67
  6. 2023Vision-Language Models as a Source of Rewardscited by 66
  7. 2016Unsupervised Perceptual Rewards for Imitation Learningcited by 64
  8. 2025ReWiND: Language-Guided Rewards Teach Robot Policies without New Demonstrationscited by 54
  9. 2023Robot Fine-Tuning Made Easy: Pre-Training Rewards and Policies for Autonomous Real-World Reinforcement Learningcited by 50
  10. 2024Code as Reward: Empowering Reinforcement Learning with VLMscited by 39
  11. 2025SARM: Stage-Aware Reward Modeling for Long Horizon Robot Manipulationcited by 39
  12. 2025Robo-Dopamine: General Process Reward Modeling for High-Precision Robotic Manipulationcited by 39
  13. 2026RoboDojo: A Unified Sim-and-Real Benchmark for Comprehensive Evaluation of Generalist Robot Manipulation Policiescited by 37
  14. 2023Robotic Offline RL from Internet Videos via Value-Function Pre-Trainingcited by 32
  15. 2024Rank2Reward: Learning Shaped Reward Functions from Passive Videocited by 30
  16. 2024Task Success is not Enough: Investigating the Use of Video-Language Models as Behavior Critics for Catching Undesirable Agent Behaviorscited by 23
  17. 2024GarmentLab: A Unified Simulation and Benchmark for Garment Manipulationcited by 20
  18. 2024VICtoR: Learning Hierarchical Vision-Instruction Correlation Rewards for Long-horizon Manipulationcited by 11
  19. 2026ProgressVLA: Progress-Guided Diffusion Policy for Vision-Language Robotic Manipulationcited by 10
  20. 2025OpenGVL -- Benchmarking Visual Temporal Progress for Data Curationcited by 9
  21. 2024ELEMENTAL: Interactive Learning from Demonstrations and Vision-Language Models for Reward Design in Roboticscited by 8
  22. 2025Subtask-Aware Visual Reward Learning from Segmented Demonstrationscited by 7
  23. 2026Recurrent Reasoning with Vision-Language Models for Estimating Long-Horizon Embodied Task Progresscited by 6
  24. 2026Generalizable Dense Reward for Long-Horizon Robotic Taskscited by 3

Built on by

Cited by year (2026 to 13 Oct)

Citing papers per year: 2026: 5.

5 papers.

  1. 2026FailBench: How Reliable are VLMs at Judging Robot Task Success?cited by 0
  2. 2026ProgressCompass: Embodied Progress Reward Models Are Lost Without the Right Contextcited by 0
  3. 2026Video2Skill: From Streaming Experience to Reusable Embodied Skillscited by 0
  4. 2026Reward-DAgger: Robot-Gated Interactive Imitation Learning with General-Purpose Progress-Based Reward Modelscited by 0
  5. 2026DiVeR: Decision-Critical Verifier Learning for VLA Test-Time Scalingcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024How Much Progress Did I Make? An Unexplored Human Feedback Signal for Teaching Robots
  2. 2026ARS: Agentic Reward System for Robot Learning
  3. 2026ProcVLM: Learning Procedure-Grounded Progress Rewards for Robotic Manipulation
  4. 2026RARM: Confidence-Gated Progress Reward Modeling for RL in Manipulation
  5. 2024PROGRESSOR: A Perceptually Guided Reward Estimator with Self-Supervised Online Refinement
  6. 2026UR-VC: Unsupervised Robotic Value Correction for Time-Derived Progress Proxies
  7. 2026World Model for Robot Learning: A Comprehensive Survey
  8. 2019Benchmark for Skill Learning from Demonstration: Impact of User Experience, Task Complexity, and Start Configuration on Performance