mlsanityData to 14 Oct 2026

PRICE the Action Chunks: Physical Relational Credit Assignment for Embodied Reinforcement Learning

arXiv 2609.38890 ↗First posted 30 Sep 2026cs.ROcited by 0

On the map: Deep reinforcement learning methods → RL for robot locomotion and manipulation also fits: Deep RL algorithms and representations

Abstract (arXiv)

Outcome-based reinforcement learning (RL) post-trains vision--language--action policies using terminal success signals, but assigns the same trajectory-level advantage to every action chunk. A failed episode can thus penalize useful early actions as if they caused the failure. Existing approaches seek finer-grained feedback through learned evaluators, adding task-specific supervision or additional model training. We explore, for the first time to our knowledge, whether physical relations across trajectories can provide action-chunk credit in embodied RL from terminal outcomes alone, without an auxiliary evaluator. The key insight is that rollouts reaching corresponding physical situations can serve as references for one another: their terminal outcomes provide evidence for assessing local progress. We introduce Physical Relations for Inferring Credit from Episodes(PRICE), with two components: (i) a physical relational graph that pools current and historical outcomes at corresponding chunk boundaries to estimate success potentials; and (ii) confidence-gated credit assignment that uses changes in these potentials to refine trajectory-level supervision. Our analysis connects oracle potential changes to the terminal-success objective and provides a finite-sample directional bound for outcome-independent evidence pools. Independent continuation tests show that PRICE's retained credits align with local progress, while experiments on LIBERO, RoboTwin 2.0, and real robots demonstrate improved task success over outcome-based baselines and faster learning.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in RL for robot locomotion and manipulation, one of 15 directions in Deep reinforcement learning methods. Builds on 35 of its 38 references that are in the map. First posted on arXiv 30 Sep 2026.

Builds on

35 of 38 references.

Closest ancestors · 1

Same direction, newest first

  1. 2026RISE: Self-Improving Robot Policy with Compositional World Model11 Feb

Other references · 34

  1. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  2. 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,887
  3. 2015High-Dimensional Continuous Control Using Generalized Advantage Estimationcited by 2,026
  4. 2023LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learningcited by 1,410
  5. 2019Advantage-Weighted Regression: Simple and Scalable Off-Policy Reinforcement Learningcited by 782
  6. 2025RoboTwin 2.0: A Scalable Data Generator and Benchmark with Strong Domain Randomization for Robust Bimanual Robotic Manipulationcited by 518
  7. 2025Group-in-Group Policy Optimization for LLM Agent Trainingcited by 497
  8. 2026Fast-WAM: Do World Action Models Need Test-time Future Imagination?cited by 349
  9. 2019DeepMDP: Learning Continuous Latent Space Models for Representation Learningcited by 207
  10. 2025VLA-RL: Towards Masterful and General Robotic Manipulation with Scalable Reinforcement Learningcited by 176
  11. 2018RUDDER: Return Decomposition for Delayed Rewardscited by 173
  12. 2025SimpleVLA-RL: Scaling VLA Training via Reinforcement Learningcited by 151
22 more
  1. 2025Interactive Post-Training for Vision-Language-Action Modelscited by 135
  2. 2024GRAPE: Generalizing Robot Policy via Preference Alignmentcited by 109
  3. 2025Reinforcement Learning with Action Chunkingcited by 107
  4. 2025WMPO: World Model-based Policy Optimization for Vision-Language-Action Modelscited by 70
  5. 2017Near Optimal Behavior via Approximate State Abstractioncited by 66
  6. 2025World-Env: Leveraging World Model as a Virtual Environment for VLA Post-Trainingcited by 59
  7. 2026WoVR: World Models as Reliable Simulators for Post-Training VLA Policies with RLcited by 50
  8. 2025TGRPO :Fine-tuning Vision-Language-Action Model via Trajectory-wise Group Relative Policy Optimizationcited by 49
  9. 2026Beyond Trajectory-Level Attribution: Graph-Based Credit Assignment for Agentic Reinforcement Learningcited by 31
  10. 2025RLinf-VLA: A Unified and Efficient Framework for Reinforcement Learning of Vision-Language-Action Modelscited by 28
  11. 2025SALT: Step-level Advantage Assignment for Long-horizon Agents via Trajectory Graphcited by 28
  12. 2025SRPO: Self-Referential Policy Optimization for Vision-Language-Action Modelscited by 27
  13. 2025RFTF: Reinforcement Fine-tuning for Embodied Agents with Temporal Feedbackcited by 20
  14. 2026Group-Graph Policy Optimization for Long-Horizon Agentic Reinforcement Learningcited by 13
  15. 2026GAGPO: Generalized Advantage Grouped Policy Optimizationcited by 9
  16. 2026BiPACE: Bisimulation-Guided Policy Optimization with Action Counterfactual Estimation for LLM Agentscited by 7
  17. 2026WAM-RL: World-Action Model Reinforcement Learning with Reconstruction Rewards and Online Video SFTcited by 4
  18. 2026HiFi-UMI: Learning Deployable Manipulation Policies from High-Fidelity UMI Data Alonecited by 4
  19. 2026Gated-BEPO: Confidence-Gated Bellman Credit Assignment for Large Language Model Agentscited by 2
  20. 2026Temporal GRPO: Beyond Trajectory-Level Credit in Vision-Language-Action Reinforcement Learningcited by 2
  21. 2026PACE: Phase-Progress-Aware Credit for Long-Horizon Embodied Manipulationcited by 1
  22. 2026Dream2Reward: Transition-Alignment Reward Models from Positive Demonstrations for Robotic Manipulationcited by 1

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2026Reconciling Process Supervision with Outcome-Based Credit in Agentic Policy Optimization
  2. 2026Dependency-Aware Reward Shaping for Agentic Reinforcement Learning
  3. 2026From Reasoning to Agentic: Credit Assignment in Reinforcement Learning for Large Language Models
  4. 2026ProCredit: From Outcome Rewards to Progress Credit in Agentic Reinforcement Learning
  5. 2026TRACE: Turn-level Reward Assignment via Credit Estimation for Long-Horizon Agents
  6. 2026From Perception to Action: An Interactive Benchmark for Vision Reasoning
  7. 2026Residual Visual Credit Optimization: Conserved Evidence Routing for Multimodal Reinforcement Learning
  8. 2026How Much, Then Where: Credit-Conserving Action-to-Token Allocation for Multi-Turn Agent Reinforcement Learning