mlsanityData to 14 Oct 2026

Temporal GRPO: Beyond Trajectory-Level Credit in Vision-Language-Action Reinforcement Learning

arXiv 2608.13026 ↗First posted 13 Aug 2026cs.ROcited by 2

On the map: LLM reasoning → Reinforcement learning for reasoning

also in 1 more
Abstract (arXiv)

Outcome-driven reinforcement learning offers a scalable way to post-train vision-language-action (VLA) policies from sparse task-success feedback. In common GRPO-based VLA post-training, one rollout-level advantage is applied to every action in the trajectory. A rollout that completes several valid stages but fails later can therefore penalize the actions that produced its earlier progress. We call this trajectory-level credit aliasing. Temporal GRPO addresses this problem by constructing detectable task stages, aligning each rollout with stage-specific action intervals, and comparing only rollouts that have entered the same stage. The resulting stage advantages are applied to their corresponding intervals in a single policy update. On RoboTwin 2.0, Temporal GRPO improves task success and sample efficiency, with consistent gains across task horizons. Controlled updates on LIBERO-Long preserve shared prerequisite stages and concentrate improvement at the first stage where rollout outcomes diverge.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Placed in Reinforcement learning for reasoning, one of 17 directions in LLM reasoning. Builds on 20 of its 30 references that are in the map. First posted on arXiv 13 Aug 2026.

Builds on

20 of 30 references.

References · 20

  1. 2023RT-2: Vision-Language-Action Models Transfer Web Knowledge to Robotic Controlcited by 3,080
  2. 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,887
  3. 2024π0: A Vision-Language-Action Flow Model for General Robot Controlcited by 2,407
  4. 2023LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learningcited by 1,410
  5. 2025GR00T N1: An Open Foundation Model for Generalist Humanoid Robotscited by 1,225
  6. 2024RDT-1B: a Diffusion Foundation Model for Bimanual Manipulationcited by 691
  7. 2025RoboTwin 2.0: A Scalable Data Generator and Benchmark with Strong Domain Randomization for Robust Bimanual Robotic Manipulationcited by 518
  8. 2024CogACT: A Foundational Vision-Language-Action Model for Synergizing Cognition and Action in Robotic Manipulationcited by 422
  9. 2025VLA-RL: Towards Masterful and General Robotic Manipulation with Scalable Reinforcement Learningcited by 176
  10. 2025ConRFT: A Reinforced Fine-tuning Method for VLA Models via Consistency Policycited by 154
  11. 2025SimpleVLA-RL: Scaling VLA Training via Reinforcement Learningcited by 151
  12. 2025Interactive Post-Training for Vision-Language-Action Modelscited by 135
8 more
  1. 2024GRAPE: Generalizing Robot Policy via Preference Alignmentcited by 109
  2. 2025TGRPO :Fine-tuning Vision-Language-Action Model via Trajectory-wise Group Relative Policy Optimizationcited by 49
  3. 2025ReinboT: Amplifying Robot Visual-Language Manipulation with Reinforcement Learningcited by 47
  4. 2025SARM: Stage-Aware Reward Modeling for Long Horizon Robot Manipulationcited by 42
  5. 2020Align-RUDDER: Learning From Few Demonstrations by Reward Redistributioncited by 36
  6. 2026RynnBrain: Open Embodied Foundation Modelscited by 35
  7. 2026LaST-R1: Reinforcing Robotic Manipulation via Adaptive Physical Latent Reasoningcited by 8
  8. 2025Reinforcement Learning with Segment Feedbackcited by 4

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026DeliveryGym: An RL Environment for Long-Horizon Embodied Agent Planning with Adaptive Curriculumcited by 2
  2. 2026PRICE the Action Chunks: Physical Relational Credit Assignment for Embodied Reinforcement Learningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Prism-GRPO: Faster VLA Policy Optimization via Splitting Same-outcome Groups
  2. 2025STARE-VLA: Progressive Stage-Aware Reinforcement for Fine-Tuning Vision-Language-Action Models
  3. 2026Learn Where Outcomes Diverge: Efficient VLA RL via Probabilistic Chunk Masking
  4. 2026PAPO-VLA: Planning-Aware Policy Optimization for Vision-Language-Action Models
  5. 2026Z-1: Efficient Reinforcement Learning for Vision-Language-Action Models
  6. 2025SRPO: Self-Referential Policy Optimization for Vision-Language-Action Models
  7. 2026StructRL: Online Structured Reinforcement Learning for Long-Horizon Vision-Language-Action Tasks
  8. 2026ACRO: Actor-Critic Rollout Orchestration