mlsanityData to 14 Oct 2026

Efficient On-policy Visual-RL via Stochastic Decoupled Policy Gradient

arXiv 2605.26478 ↗First posted 26 May 2026cs.RO cs.AI cs.CVcited by 1

On the map: Deep reinforcement learning methods → RL for robot locomotion and manipulation

Abstract (arXiv)

We present the stochastic decoupled policy gradient (SDPG), a lightweight visual reinforcement learning (RL) method that trains diverse visuomotor control policies end-to-end within a few hours on a single NVIDIA RTX 4080 GPU. SDPG estimates policy gradients via random perturbations of trajectory rollouts, requiring orders of magnitude fewer batch-rendered environments and substantially reducing compute and memory overhead. On visual MuJoCo benchmarks, SDPG consistently outperforms baseline methods in training time, memory usage, and rewards. Finally, to support future research, we introduce a suite of realistic visual robotics benchmarks spanning dexterous manipulation, challenging locomotion, and demonstrate effective sim-to-real transfer on physical hardware.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Placed in RL for robot locomotion and manipulation, one of 15 directions in Deep reinforcement learning methods. Builds on 41 of its 54 references that are in the map. First posted on arXiv 26 May 2026.

Builds on

41 of 54 references.

Closest ancestors · 5

Same direction, newest first

  1. 2025RSL-RL: A Learning Library for Robotics Research13 Sep
  2. 2025Steering Your Diffusion Policy with Latent Space Reinforcement Learning18 Jun
  3. 2025Accelerating Visual-Policy Learning through Parallel Differentiable Simulation15 May
  4. 2024When Should We Prefer State-to-Visual DAgger Over Visual Reinforcement Learning?18 Dec
  5. 2024Stabilizing Reinforcement Learning in Differentiable Multiphysics Simulation16 Dec

Other references · 36

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2017Proximal Policy Optimization Algorithmscited by 15,215
  3. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,885
  4. 2015Continuous control with deep reinforcement learningcited by 3,407
  5. 2013Playing Atari with Deep Reinforcement Learningcited by 3,274
  6. 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,258
  7. 2019Dream to Control: Learning Behaviors by Latent Imaginationcited by 1,653
  8. 2018Soft Actor-Critic Algorithms and Applicationscited by 1,240
  9. 2021Isaac Gym: High Performance GPU-Based Physics Simulation For Robot Learningcited by 998
  10. 2020CURL: Contrastive Unsupervised Representations for Reinforcement Learningcited by 760
  11. 2020Learning Quadrupedal Locomotion over Challenging Terraincited by 604
  12. 2020Image Augmentation Is All You Need: Regularizing Deep Reinforcement Learning from Pixelscited by 585
24 more
  1. 2017Evolution Strategies as a Scalable Alternative to Reinforcement Learningcited by 574
  2. 2024Revisiting Feature Prediction for Learning Visual Representations from Videocited by 531
  3. 2021Learning to Walk in Minutes Using Massively Parallel Deep Reinforcement Learningcited by 512
  4. 2019Learning agile and dynamic motor skills for legged robotscited by 504
  5. 2022Temporal Difference Learning for Model Predictive Controlcited by 397
  6. 2022Learning robust perceptive locomotion for quadrupedal robots in the wildcited by 382
  7. 2021Mastering Visual Continuous Control: Improved Data-Augmented Reinforcement Learningcited by 372
  8. 2017Asymmetric Actor Critic for Image-Based Robot Learningcited by 238
  9. 2016Continuous Deep Q-Learning with Model-based Accelerationcited by 157
  10. 2022Legged Locomotion in Challenging Terrains using Egocentric Visioncited by 151
  11. 2023Policy Representation via Diffusion Probability Model for Reinforcement Learningcited by 112
  12. 2022Accelerated Policy Learning with Parallel Differentiable Simulationcited by 104
  13. 2022Do Differentiable Simulators Give Better Policy Gradients?cited by 89
  14. 2018Simple random search provides a competitive approach to reinforcement learningcited by 75
  15. 2022Predictive Sampling: Real-time Behaviour Synthesis with MuJoCocited by 47
  16. 2024Back to Newton's Laws: Learning Vision-based Agile Flight via Differentiable Physicscited by 45
  17. 2024DextrAH-G: Pixels-to-Action Dexterous Arm-Hand Grasping with Geometric Fabricscited by 42
  18. 2024Learning Quadruped Locomotion Using Differentiable Simulationcited by 32
  19. 2025Parkour in the Wild: Learning a General and Extensible Agile Locomotion Policy Using Multi-expert Distillation and RL Fine-tuningcited by 26
  20. 2016Path Integral Guided Policy Searchcited by 15
  21. 2024Learning Deployable Locomotion Control via Differentiable Simulationcited by 8
  22. 2025Learning on the Fly: Rapid Policy Adaptation via Differentiable Simulationcited by 8
  23. 2025End-to-end RL Improves Dexterous Grasping Policiescited by 5
  24. 2025Distilling Realizable Students from Unrealizable Teacherscited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2026: 1.

1 paper.

  1. 2026Accelerating Visual Policy Learning with Sampling-Based Model Predictive Controlcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025Opening the Sim-to-Real Door for Humanoid Pixel-to-Action Policy Transfer
  2. 2026Squint: Fast Visual Reinforcement Learning for Sim-to-Real Robotics
  3. 2024Consistency Policy: Accelerated Visuomotor Policies via Consistency Distillation
  4. 2025Pre-trained Visual Representations Generalize Where it Matters in Model-Based Reinforcement Learning
  5. 2018Reinforcement and Imitation Learning for Diverse Visuomotor Skills
  6. 2026GzDRL: Reproducible and Scalable Deep Reinforcement Learning with Gazebo
  7. 2026V-Simba: Unleashing the Architectural Potential of RL in Visual Continuous Control
  8. 2025Exploration and Adaptation in Non-Stationary Tasks with Diffusion Policies