mlsanityData to 7 Oct 2026

SERL: A Software Suite for Sample-Efficient Robotic Reinforcement Learning

arXiv 2401.16013 ↗First posted 29 Jan 2024cs.RO cs.AIcited by 113

Not on the map. Why?

Cited by 37 in Vision-language-action models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

In recent years, significant progress has been made in the field of robotic reinforcement learning (RL), enabling methods that handle complex image observations, train in the real world, and incorporate auxiliary data, such as demonstrations and prior experience. However, despite these advances, robotic RL remains hard to use. It is acknowledged among practitioners that the particular implementation details of these algorithms are often just as important (if not more so) for performance as the choice of algorithm. We posit that a significant challenge to widespread adoption of robotic RL, as well as further development of robotic RL methods, is the comparative inaccessibility of such methods. To address this challenge, we developed a carefully implemented library containing a sample efficient off-policy deep RL method, together with methods for computing rewards and resetting the environment, a high-quality controller for a widely-adopted robot, and a number of challenging example tasks. We provide this library as a resource for the community, describe its design choices, and present experimental results. Perhaps surprisingly, we find that our implementation can achieve very efficient learning, acquiring policies for PCB board assembly, cable routing, and object relocation between 25 to 50 minutes of training per policy on average, improving over state-of-the-art results reported for similar tasks in the literature. These policies achieve perfect or near-perfect success rates, extreme robustness even under perturbations, and exhibit emergent recovery and correction behaviors. We hope that these promising results and our high-quality open-source implementation will provide a tool for the robotics community to facilitate further developments in robotic RL. Our code, documentation, and videos can be found at https://serl-robot.github.io/

Key facts. Cited by 113 papers in the map (data through 7 Oct 2026). Builds on 48 of its 69 references that are in the map. First posted on arXiv 29 Jan 2024.

Builds on

48 of 69 references.

References · 48

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2018Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actorcited by 4,635
  3. 2013Playing Atari with Deep Reinforcement Learningcited by 3,211
  4. 2018Addressing Function Approximation Error in Actor-Critic Methodscited by 2,302
  5. 2016OpenAI Gymcited by 1,684
  6. 2019Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learningcited by 1,168
  7. 2015End-to-End Training of Deep Visuomotor Policiescited by 847
  8. 2017Learning Complex Dexterous Manipulation with Deep Reinforcement Learning and Demonstrationscited by 685
  9. 2019RLBench: The Robot Learning Benchmark & Learning Environmentcited by 669
  10. 2020AWAC: Accelerating Online Reinforcement Learning with Offline Datasetscited by 647
  11. 2022MineDojo: Building Open-Ended Embodied Agents with Internet-Scale Knowledgecited by 469
  12. 2022VIP: Towards Universal Visual Reward and Representation via Value-Implicit Pre-Trainingcited by 384
36 more
  1. 2022DayDreamer: World Models for Physical Robot Learningcited by 372
  2. 2023Efficient Online Reinforcement Learning with Offline Datacited by 332
  3. 2016Learning Hand-Eye Coordination for Robotic Grasping with Deep Learning and Large-Scale Data Collectioncited by 312
  4. 2017Dex-Net 2.0: Deep Learning to Plan Robust Grasps with Synthetic Point Clouds and Analytic Grasp Metricscited by 259
  5. 2018Residual Reinforcement Learning for Robot Controlcited by 242
  6. 2023Orbit: A Unified Simulation Framework for Interactive Robot Learning Environmentscited by 239
  7. 2021MT-Opt: Continuous Multi-Task Robotic Reinforcement Learning at Scalecited by 226
  8. 2019Deep Dynamics Models for Learning Dexterous Manipulationcited by 200
  9. 2023LIV: Language-Image Representations and Rewards for Robotic Controlcited by 180
  10. 2023Vision-Language Models as Success Detectorscited by 138
  11. 2023FurnitureBench: Reproducible Real-World Benchmark for Long-Horizon Complex Manipulationcited by 132
  12. 2020The Ingredients of Real-World Robotic Reinforcement Learningcited by 111
  13. 2020Offline Reinforcement Learning from Images with Latent Space Modelscited by 94
  14. 2021Reset-Free Reinforcement Learning via Multi-Task Learning: Learning Dexterous Manipulation Behaviors without Human Interventioncited by 90
  15. 2017Leave no Trace: Learning to Reset for Safe and Autonomous Reinforcement Learningcited by 86
  16. 2021d3rlpy: An Offline Deep Reinforcement Learning Librarycited by 84
  17. 2018Dexterous Manipulation with Deep Reinforcement Learning: Efficient, General, and Low-Costcited by 79
  18. 2024FMB: a Functional Manipulation Benchmark for Generalizable Robotic Learningcited by 79
  19. 2019ROBEL: Robotics Benchmarks for Learning with Low-Cost Robotscited by 70
  20. 2018Variational Inverse Control with Events: A General Framework for Data-Driven Reward Definitioncited by 66
  21. 2017Data-efficient Deep Reinforcement Learning for Dexterous Manipulationcited by 61
  22. 2019Data Efficient Reinforcement Learning for Legged Robotscited by 60
  23. 2019Deep Reinforcement Learning for Industrial Insertion Tasks with Visual Inputs and Natural Rewardscited by 58
  24. 2021Offline Meta-Reinforcement Learning for Industrial Insertioncited by 37
  25. 2018A Practical Approach to Insertion with Variable Socket Position Using Deep Reinforcement Learningcited by 35
  26. 2019Reinforcement Learning on Variable Impedance Controller for High-Precision Robotic Assemblycited by 33
  27. 2020Learning to Play Table Tennis From Scratch using Muscular Robotscited by 31
  28. 2023Self-Improving Robots: End-to-End Autonomous Visuomotor Reinforcement Learningcited by 29
  29. 2021Robust Multi-Modal Policies for Industrial Assembly via Reinforcement Learning and Demonstrations: A Large-Scale Studycited by 24
  30. 2021MURAL: Meta-Learning Uncertainty-Aware Rewards for Outcome-Driven Reinforcement Learningcited by 24
  31. 2023RLIF: Interactive Imitation Learning as Reinforcement Learningcited by 24
  32. 2022Lyapunov Design for Robust and Efficient Robotic Reinforcement Learningcited by 20
  33. 2022When to Ask for Help: Proactive Interventions in Autonomous Reinforcement Learningcited by 20
  34. 2021InsertionNet -- A Scalable Solution for Insertioncited by 15
  35. 2023REBOOT: Reuse Data for Bootstrapping Efficient Real-World Dexterous Manipulationcited by 12
  36. 2020Sample-efficient Reinforcement Learning in Robotic Table Tenniscited by 10

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 20; 2025: 35; 2026: 58.

113 papers.

Sort:

By area:

  1. 2024Diffusion Policy Policy Optimizationcited by 255
  2. 2025VLA-RL: Towards Masterful and General Robotic Manipulation with Scalable Reinforcement Learningcited by 166
  3. 2025Steering Your Diffusion Policy with Latent Space Reinforcement Learningcited by 149
  4. 2025ConRFT: A Reinforced Fine-tuning Method for VLA Models via Consistency Policycited by 139
  5. 2025GraspVLA: a Grasping Foundation Model Pre-trained on Billion-scale Synthetic Action Datacited by 120
  6. 2025Improving Vision-Language-Action Model with Online Reinforcement Learningcited by 107
  7. 2025Vision-Language-Action Models for Robotics: A Review Towards Real-World Applicationscited by 82
  8. 2024From Imitation to Refinement -- Residual RL for Precise Assemblycited by 80
  9. 2025Self-Improving Vision-Language-Action Models with Data Generation via Residual RLcited by 72
  10. 2025A Vision-Language-Action-Critic Model for Robotic Real-World Reinforcement Learningcited by 66
  11. 2024Prediction with Action: Visual Policy Learning via Joint Denoising Processcited by 64
  12. 2024Efficient Online Reinforcement Learning Fine-Tuning Need Not Retain Offline Datacited by 58
38 more
  1. 2025RL-100: Performant Robotic Manipulation with Real-World Reinforcement Learningcited by 47
  2. 2026RL Token: Bootstrapping Online RL with Vision-Language-Action Modelscited by 43
  3. 2024RLDG: Robotic Generalist Policy Distillation via Reinforcement Learningcited by 42
  4. 2025Residual Off-Policy RL for Finetuning Behavior Cloning Policiescited by 42
  5. 2025GR-RL: Going Dexterous and Precise for Long-Horizon Robotic Manipulationcited by 41
  6. 2025GWM: Towards Scalable Gaussian World Models for Robotic Manipulationcited by 37
  7. 2024R+X: Retrieval and Execution from Everyday Human Videoscited by 36
  8. 2025RaC: Robot Learning for Long-Horizon Tasks by Scaling Recovery and Correctioncited by 35
  9. 2024Autonomous Improvement of Instruction Following Skills via Foundation Modelscited by 32
  10. 2024Hybrid Inverse Reinforcement Learningcited by 31
  11. 2026ENPIRE: Agentic Robot Policy Self-Improvement in the Real Worldcited by 31
  12. 2024MENTOR: Mixture-of-Experts Network with Task-Oriented Perturbation for Visual Reinforcement Learningcited by 20
  13. 2026SOP: A Scalable Online Post-Training System for Vision-Language-Action Modelscited by 19
  14. 2024Continuous Control with Coarse-to-fine Reinforcement Learningcited by 15
  15. 2026Learning While Deploying: Fleet-Scale Reinforcement Learning for Generalist Robot Policiescited by 12
  16. 2024Unsupervised-to-Online Reinforcement Learningcited by 11
  17. 2025Rethinking Bimanual Robotic Manipulation: Learning with Decoupled Interaction Frameworkcited by 11
  18. 2026chi0: Resource-Aware Robust Manipulation via Taming Distributional Inconsistenciescited by 11
  19. 2024MILES: Making Imitation Learning Easy with Self-Supervisioncited by 9
  20. 2026Beyond Imitation: Reinforcement Learning-Based Sim-Real Co-Training for VLA Modelscited by 8
  21. 2026From Prior to Pro: Efficient Skill Mastery via Distribution Contractive RL Finetuningcited by 8
  22. 2024On-Robot Reinforcement Learning with Goal-Contrastive Rewardscited by 7
  23. 2025SPEQ: Offline Stabilization Phases for Efficient Q-Learning in High Update-To-Data Ratio Reinforcement Learningcited by 7
  24. 2025SLAC: Safe and Efficient Real-Robot Reinforcement Learning via Unsupervised Simulation Pre-Trainingcited by 7
  25. 2026AnyCamVLA: Zero-Shot Camera Adaptation for Viewpoint Robust Vision-Language-Action Modelscited by 7
  26. 2026LaST-R1: Reinforcing Robotic Manipulation via Adaptive Physical Latent Reasoningcited by 7
  27. 2024Automatic Environment Shaping is the Next Frontier in RLcited by 6
  28. 2025Uncertainty Comes for Free: Human-in-the-Loop Policies with Diffusion Modelscited by 6
  29. 2025SIME: Enhancing Policy Self-Improvement with Modal-level Explorationcited by 6
  30. 2025CrayonRobo: Object-Centric Prompt-Driven Vision-Language-Action Model for Robotic Manipulationcited by 6
  31. 2025Multistep Quasimetric Learning for Scalable Goal-conditioned Reinforcement Learningcited by 6
  32. 2024PointPatchRL -- Masked Reconstruction Improves Reinforcement Learning on Point Cloudscited by 5
  33. 2025Learning to See and Act: Task-Aware Virtual View Exploration for Robotic Manipulationcited by 5
  34. 2025SOE: Sample-Efficient Robot Policy Self-Improvement via On-Manifold Explorationcited by 5
  35. 2025Human-in-the-loop Online Rejection Sampling for Robotic Manipulationcited by 5
  36. 2026IG-RFT: An Interaction-Guided RL Framework for VLA Models in Long-Horizon Robotic Manipulationcited by 5
  37. 2026You've Got a Golden Ticket: Improving Generative Robot Policies With A Single Noise Vectorcited by 5
  38. 2024Learning Diverse Robot Striking Motions with Diffusion Models and Kinematically Constrained Gradient Guidancecited by 4

50 most cited of 113 All 113 (JSON)