# Benchmarking Model Predictive Control and Reinforcement Learning Based Control for Legged Robot Locomotion in MuJoCo Simulation

arXiv: https://arxiv.org/abs/2501.16590 · HTML: https://mlsanity.com/paper/2501.16590/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Placed in RL for robot locomotion and manipulation, one of 15 directions in Deep reinforcement learning methods.
- Builds on 28 of its 60 references that are in the map.
- First posted on arXiv 28 Jan 2025.

## Placed in

- Deep reinforcement learning methods / [RL for robot locomotion and manipulation](https://mlsanity.com/deeprl/rl-for-robot-locomotion-and-manipulation/)

## Builds on (most cited first, 28 of 28)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Playing Atari with Deep Reinforcement Learning](https://mlsanity.com/paper/1312.5602/) (2013)
- [Trust Region Policy Optimization](https://mlsanity.com/paper/1502.05477/) (2015)
- [Optuna: A Next-generation Hyperparameter Optimization Framework](https://mlsanity.com/paper/1907.10902/) (2019)
- [OpenAI Gym](https://mlsanity.com/paper/1606.01540/) (2016)
- [Soft Actor-Critic Algorithms and Applications](https://mlsanity.com/paper/1812.05905/) (2018)
- [Sim-to-Real Transfer of Robotic Control with Dynamics Randomization](https://mlsanity.com/paper/1710.06537/) (2017)
- [Learning Quadrupedal Locomotion over Challenging Terrain](https://mlsanity.com/paper/2010.11251/) (2020)
- [Deep Reinforcement Learning for Robotic Manipulation with Asynchronous Off-Policy Updates](https://mlsanity.com/paper/1610.00633/) (2016)
- [Rapid Locomotion via Reinforcement Learning](https://mlsanity.com/paper/2205.02824/) (2022)
- [Preparing for the Unknown: Learning a Universal Policy with Online System Identification](https://mlsanity.com/paper/1702.02453/) (2017)
- [Reinforcement Learning for Robust Parameterized Locomotion Control of Bipedal Robots](https://mlsanity.com/paper/2103.14295/) (2021)
- [Reaching the Limit in Autonomous Racing: Optimal Control versus Reinforcement Learning](https://mlsanity.com/paper/2310.10943/) (2023)
- [Why Generalization in RL is Difficult: Epistemic POMDPs and Implicit Partial Observability](https://mlsanity.com/paper/2107.06277/) (2021)
- [Data Efficient Reinforcement Learning for Legged Robots](https://mlsanity.com/paper/1907.03613/) (2019)
- [RLOC: Terrain-Aware Legged Locomotion using Reinforcement Learning and Optimal Control](https://mlsanity.com/paper/2012.03094/) (2020)
- [Predictive Sampling: Real-time Behaviour Synthesis with MuJoCo](https://mlsanity.com/paper/2212.00541/) (2022)
- [Learning Whole-body Manipulation for Quadrupedal Robot](https://mlsanity.com/paper/2308.16820/) (2023)
- [Vision-Guided Quadrupedal Locomotion in the Wild with Multi-Modal Delay Randomization](https://mlsanity.com/paper/2109.14549/) (2021)
- [Learning-based legged locomotion; state of the art and future perspectives](https://mlsanity.com/paper/2406.01152/) (2024)
- [Feedback MPC for Torque-Controlled Legged Robots](https://mlsanity.com/paper/1905.06144/) (2019)
- [Benchmarking the Sim-to-Real Gap in Cloth Manipulation](https://mlsanity.com/paper/2310.09543/) (2023)
- [Collision-Free MPC for Legged Robots in Static and Dynamic Scenes](https://mlsanity.com/paper/2103.13987/) (2021)
- [Real-Time Motion Planning of Legged Robots: A Model Predictive Control Approach](https://mlsanity.com/paper/1710.04029/) (2017)
- [Learning Agile Locomotion and Adaptive Behaviors via RL-augmented MPC](https://mlsanity.com/paper/2310.09442/) (2023)
- [Generalization in Transfer Learning](https://mlsanity.com/paper/1909.01331/) (2019)
- [Model Predictive Control with Environment Adaptation for Legged Locomotion](https://mlsanity.com/paper/2105.05998/) (2021)
- [Test and Evaluation of Quadrupedal Walking Gaits through Sim2Real Gap Quantification](https://mlsanity.com/paper/2201.01323/) (2022)

## Built on by (most cited first, 3 of 3)

- [MIMIC-MJX: Neuromechanical Emulation of Animal Behavior](https://mlsanity.com/paper/2511.20532/) (2025)
- [Isaac Sim-to-Real: Reinforcement Learning based Locomotion for Quadrupeds](https://mlsanity.com/paper/2607.18135/) (2026)
- [Towards Torque-Driven Reinforcement Learning for Quadruped Locomotion](https://mlsanity.com/paper/2607.18365/) (2026)

## Abstract (arXiv)

Model Predictive Control (MPC) and Reinforcement Learning (RL) are two prominent strategies for controlling legged robots, each with unique strengths. RL learns control policies through system interaction, adapting to various scenarios, whereas MPC relies on a predefined mathematical model to solve optimization problems in real-time. Despite their widespread use, there is a lack of direct comparative analysis under standardized conditions. This work addresses this gap by benchmarking MPC and RL controllers on a Unitree Go1 quadruped robot within the MuJoCo simulation environment, focusing on a standardized task-straight walking at a constant velocity. Performance is evaluated based on disturbance rejection, energy efficiency, and terrain adaptability. The results show that RL excels in handling disturbances and maintaining energy efficiency but struggles with generalization to new terrains due to its dependence on learned policies tailored to specific environments. In contrast, MPC shows enhanced recovery capabilities from larger perturbations by leveraging its optimization-based approach, allowing for a balanced distribution of control efforts across the robot's joints. The results provide a clear understanding of the advantages and limitations of both RL and MPC, offering insights into selecting an appropriate control strategy for legged robotic applications.