mlsanityData to 7 Oct 2026

Prism-GRPO: Faster VLA Policy Optimization via Splitting Same-outcome Groups

arXiv 2608.17423 ↗First posted 18 Aug 2026cs.RO cs.LGcited by 2

On the map: LLM reasoning → Reinforcement learning for reasoning

also in 1 more
Abstract (arXiv)

GRPO is increasingly used for reinforcement learning of vision-language-action (VLA) policies because, unlike PPO, it does not require training a critic. This simplification comes with a sampling cost: group-relative advantages require multiple rollouts from each scene. Under binary success rewards, groups whose rollouts all succeed or all fail have zero advantage and are discarded by dynamic sampling. These groups are especially common early in training, when most rollouts fail, wasting much of the expensive robotic rollout budget. We introduce Prism-GRPO, which augments binary outcome reward with a weighted trajectory-level execution-quality score. By splitting same-outcome groups into a quality spectrum, Prism-GRPO recovers training signal while ensuring that every success still outranks every failure. Quality scores can be derived from simulator contacts, executed actions, or visual observations, avoiding task-specific progress rewards. We prove that Prism-GRPO never increases the probability that a sampled group is discarded for having zero advantages, and derive a gradient-alignment condition under which its combined update remains a local ascent direction for task success. Across four RoboTwin tasks spanning different horizons and coordination patterns, Prism-GRPO improves success and quality at matched rollout budgets and reaches target success rates with up to 56% fewer rollouts. It also suppresses a reward-hacking shortcut, with the cleaner behavior transferring under direct deployment to a real robot. Through ablations, we show consistent gains across contact-, smoothness-, and VLM-derived quality signals.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Placed in Reinforcement learning for reasoning, one of 17 directions in LLM reasoning. Builds on 27 of its 35 references that are in the map. First posted on arXiv 18 Aug 2026.

Builds on

27 of 35 references.

Closest ancestors · 8

Same direction, newest first

  1. 2026CAST: Non-Privileged Clipped Asymmetric Self-Teaching with Advantage Flipping for GRPO29 May
  2. 2025No Prompt Left Behind: Exploiting Zero-Variance Prompts in LLM Reinforcement Learning via Entropy-Guided Advantage Shaping26 Sep
  3. 2025GTPO: Stabilizing Group Relative Policy Optimization via Gradient and Entropy Control5 Aug
  4. 2025EDGE-GRPO: Entropy-Driven GRPO with Guided Error Correction for Advantage Diversity29 Jul
  5. 2025Spurious Rewards: Rethinking Training Signals in RLVR12 Jun
  6. 2025The Entropy Mechanism of Reinforcement Learning for Reasoning Language Models28 May
  7. 2025Training Language Models to Reason Efficiently6 Feb
  8. 2017Proximal Policy Optimization Algorithms20 Jul

Other references · 19

  1. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
  2. 2024OpenVLA: An Open-Source Vision-Language-Action Modelcited by 2,777
  3. 2024π0: A Vision-Language-Action Flow Model for General Robot Controlcited by 2,297
  4. 2025π0.5: a Vision-Language-Action Model with Open-World Generalizationcited by 1,653
  5. 2025Fine-Tuning Vision-Language-Action Models: Optimizing Speed and Successcited by 844
  6. 2025RoboTwin 2.0: A Scalable Data Generator and Benchmark with Strong Domain Randomization for Robust Bimanual Robotic Manipulationcited by 477
  7. 2025SimpleVLA-RL: Scaling VLA Training via Reinforcement Learningcited by 141
  8. 2018Adapting Auxiliary Losses Using Gradient Similaritycited by 98
  9. 2025TGRPO :Fine-tuning Vision-Language-Action Model via Trajectory-wise Group Relative Policy Optimizationcited by 47
  10. 2026RoboReward: General-Purpose Vision-Language Reward Models for Roboticscited by 46
  11. 2025RLinf-VLA: A Unified and Efficient Framework for Reinforcement Learning of Vision-Language-Action Modelscited by 26
  12. 2025STARE-VLA: Progressive Stage-Aware Reinforcement for Fine-Tuning Vision-Language-Action Modelscited by 24
7 more
  1. 2026LLM-as-a-Verifier: A General-Purpose Verification Frameworkcited by 21
  2. 2021Hierarchical Potential-based Reward Shaping from Task Specificationscited by 7
  3. 2026SmoothVLA: Aligning Vision-Language-Action Models with Physical Constraints via Intrinsic Smoothness Optimizationcited by 3
  4. 2026PolicyTrim: Boosting Intrinsic Policy Efficiency of Vision-Language-Action Modelscited by 2
  5. 2026Z-1: Efficient Reinforcement Learning for Vision-Language-Action Modelscited by 2
  6. 2026Multimodal Reward Hacking in Reinforcement Learningcited by 2
  7. 2026Vision-Language-Action Jump-Starting for Reinforcement Learning Robotic Agentscited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 2.

2 papers.

  1. 2026Save Your Saturated Data: Learning Beyond Reward Saturation in Group-Based RLcited by 0
  2. 2026When KL Regularization Misfires in Group Policy Optimizationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Learn Where Outcomes Diverge: Efficient VLA RL via Probabilistic Chunk Masking
  2. 2026Temporal GRPO: Beyond Trajectory-Level Credit in Vision-Language-Action Reinforcement Learning
  3. 2026GRPO-TTA: Test-Time Visual Tuning for Vision-Language Models via GRPO-Driven Reinforcement Learning
  4. 2025WMPO: World Model-based Policy Optimization for Vision-Language-Action Models
  5. 2026PAPO-VLA: Planning-Aware Policy Optimization for Vision-Language-Action Models
  6. 2025SRPO: Self-Referential Policy Optimization for Vision-Language-Action Models
  7. 2025TIC-GRPO: Provable and Efficient Optimization for Reinforcement Learning from Human Feedback
  8. 2026GROW: Aligning GRPO with State-Action Modeling for Open-World VLM Agents