mlsanityData to 14 Oct 2026

A Real-World Quadrupedal Locomotion Benchmark for Offline Reinforcement Learning

arXiv 2309.16718 ↗First posted 13 Sep 2023cs.RO cs.LGcited by 3

On the map: Offline RL and imitation learning from data → Real-world robot offline RL

Abstract (arXiv)

Online reinforcement learning (RL) methods are often data-inefficient or unreliable, making them difficult to train on real robotic hardware, especially quadruped robots. Learning robotic tasks from pre-collected data is a promising direction. Meanwhile, agile and stable legged robotic locomotion remains an open question in their general form. Offline reinforcement learning (ORL) has the potential to make breakthroughs in this challenging field, but its current bottleneck lies in the lack of diverse datasets for challenging realistic tasks. To facilitate the development of ORL, we benchmarked 11 ORL algorithms in the realistic quadrupedal locomotion dataset. Such dataset is collected by the classic model predictive control (MPC) method, rather than the model-free online RL method commonly used by previous benchmarks. Extensive experimental results show that the best-performing ORL algorithms can achieve competitive performance compared with the model-free RL, and even surpass it in some tasks. However, there is still a gap between the learning-based methods and MPC, especially in terms of stability and rapid adaptation. Our proposed benchmark will serve as a development platform for testing and evaluating the performance of ORL algorithms in real-world legged locomotion tasks.

Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Placed in Real-world robot offline RL, one of 13 directions in Offline RL and imitation learning from data. Builds on 26 of its 38 references that are in the map. First posted on arXiv 13 Sep 2023.

Builds on

26 of 38 references.

Closest ancestors · 1

Same direction, newest first

  1. 2023Benchmarking Offline Reinforcement Learning on Real-Robot Hardware28 Jul

Other references · 25

  1. 2020Offline Reinforcement Learning: Tutorial, Review, and Perspectives on Open Problemscited by 1,920
  2. 2020Conservative Q-Learning for Offline Reinforcement Learningcited by 1,809
  3. 2020D4RL: Datasets for Deep Data-Driven Reinforcement Learningcited by 1,427
  4. 2021Offline Reinforcement Learning with Implicit Q-Learningcited by 1,293
  5. 2018Off-Policy Deep Reinforcement Learning without Explorationcited by 1,241
  6. 2019Meta-World: A Benchmark and Evaluation for Multi-Task and Meta Reinforcement Learningcited by 1,204
  7. 2021A Minimalist Approach to Offline Reinforcement Learningcited by 899
  8. 2019Stabilizing Off-Policy Q-Learning via Bootstrapping Error Reductioncited by 840
  9. 2020AWAC: Accelerating Online Reinforcement Learning with Offline Datasetscited by 671
  10. 2019Behavior Regularized Offline Reinforcement Learningcited by 662
  11. 2020MOPO: Model-based Offline Policy Optimizationcited by 636
  12. 2020Learning Quadrupedal Locomotion over Challenging Terraincited by 604
13 more
  1. 2021COMBO: Conservative Offline Model-Based Policy Optimizationcited by 370
  2. 2020Critic Regularized Regressioncited by 275
  3. 2020Learning Agile Robotic Locomotion Skills by Imitating Animalscited by 251
  4. 2022Walk These Ways: Tuning Robot Control for Generalization with Multiplicity of Behaviorcited by 138
  5. 2020PLAS: Latent Action Space for Offline Reinforcement Learningcited by 133
  6. 2021Continual World: A Robotic Benchmark For Continual Reinforcement Learningcited by 116
  7. 2019Benchmarking Batch Deep Reinforcement Learning Algorithmscited by 111
  8. 2019ROBEL: Robotics Benchmarks for Learning with Low-Cost Robotscited by 72
  9. 2023DreamWaQ: Learning Robust Quadrupedal Locomotion With Implicit Terrain Imagination via Deep Reinforcement Learningcited by 66
  10. 2018Benchmarking Reinforcement Learning Algorithms on Real-World Robotscited by 65
  11. 2021NeoRL: A Near Real-World Benchmark for Offline Reinforcement Learningcited by 63
  12. 2016Learning Locomotion Skills Using DeepRL: Does the Choice of Action Space Matter?cited by 48
  13. 2019REPLAB: A Reproducible Low-Cost Arm Benchmark Platform for Robotic Learningcited by 17

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 1; 2025: 1; 2026: 1.

3 papers.

  1. 2024Offline Adaptation of Quadruped Locomotion using Diffusion Modelscited by 3
  2. 2026Isaac Sim-to-Real: Reinforcement Learning based Locomotion for Quadrupedscited by 2
  3. 2025Constrained Reinforcement Learning for Unstable Point-Feet Bipedal Locomotion Applied to the Bolt Robotcited by 1

Similar, not cited

Close in wording, no citation either way

  1. 2024Reinforcement Learning For Quadrupedal Locomotion: Current Advancements And Future Perspectives
  2. 2025Gait in Eight: Efficient On-Robot Learning for Omnidirectional Quadruped Locomotion
  3. 2025Learning to Walk With Less: A Dyna-Style Approach to Quadrupedal Locomotion
  4. 2025GRoQ-LoCO: Generalist and Robot-agnostic Quadruped Locomotion Control using Offline Datasets
  5. 2025Learning More With Less: Sample Efficient Model-Based RL for Loco-Manipulation
  6. 2025A Framework for Deploying Learning-based Quadruped Loco-Manipulation ↗
  7. 2026Towards Torque-Driven Reinforcement Learning for Quadruped Locomotion
  8. 2025Training Directional Locomotion for Quadrupedal Low-Cost Robotic Systems via Deep Reinforcement Learning