mlsanityData to 14 Oct 2026

Extending Group Relative Policy Optimization to Continuous Control: A Theoretical Framework for Robotic Reinforcement Learning

arXiv 2507.19555 ↗First posted 25 Jul 2025cs.RO cs.AIcited by 3

On the map: Deep reinforcement learning methods → RL for robot locomotion and manipulation

Abstract (arXiv)

Group Relative Policy Optimization (GRPO) has shown promise in discrete action spaces by eliminating value function dependencies through group-based advantage estimation. However, its application to continuous control remains unexplored, limiting its utility in robotics where continuous actions are essential. This paper presents a theoretical framework extending GRPO to continuous control environments, addressing challenges in high-dimensional action spaces, sparse rewards, and temporal dynamics. Our approach introduces trajectory-based policy clustering, state-aware advantage estimation, and regularized policy updates designed for robotic applications. We provide theoretical analysis of convergence properties and computational complexity, establishing a foundation for future empirical validation in robotic systems including locomotion and manipulation tasks.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Placed in RL for robot locomotion and manipulation, one of 15 directions in Deep reinforcement learning methods. Builds on 8 of its 13 references that are in the map. First posted on arXiv 25 Jul 2025.

Builds on

8 of 13 references.

References · 8

  1. 2017Proximal Policy Optimization Algorithmscited by 15,215
  2. 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 9,379
  3. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  4. 2015Trust Region Policy Optimizationcited by 2,580
  5. 2021Isaac Gym: High Performance GPU-Based Physics Simulation For Robot Learningcited by 998
  6. 2015End-to-End Training of Deep Visuomotor Policiescited by 875
  7. 2019Stabilizing Off-Policy Q-Learning via Bootstrapping Error Reductioncited by 840
  8. 2016Benchmarking Deep Reinforcement Learning for Continuous Controlcited by 371

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 1; 2026: 2.

3 papers.

  1. 2025Learning Without Critics? Revisiting GRPO in Classical Reinforcement Learning Environmentscited by 6
  2. 2026Consensus Group Relative Policy Optimization for Text Generationcited by 1
  3. 2026Learning to Trigger: Reinforcement Learning at the Large Hadron Collidercited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026GR2PO: Group Relative Return Policy Optimization for Continuous Robot Control
  2. 2025TGRPO :Fine-tuning Vision-Language-Action Model via Trajectory-wise Group Relative Policy Optimization
  3. 2026Discrete Action Space as a Prerequisite for GRPO Convergence in Small-Model Continuous Control
  4. 2026Constrained Group Relative Policy Optimization
  5. 2025Revisiting Group Relative Policy Optimization: Insights into On-Policy and Off-Policy Training
  6. 2025TD-GRPC: Temporal Difference Learning with Group Relative Policy Constraint for Humanoid Locomotion
  7. 2026Back to the Definition: Estimating Step-Level Advantages via Trajectory Graphs for Agentic Reinforcement Learning
  8. 2026Hierarchy-of-Groups Policy Optimization for Long-Horizon Agentic Tasks