# RL-Augmented MPC for Non-Gaited Legged and Hybrid Locomotion

arXiv: https://arxiv.org/abs/2603.10878 · HTML: https://mlsanity.com/paper/2603.10878/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in RL for robot locomotion and manipulation, one of 15 directions in Deep reinforcement learning methods.
- Builds on 20 of its 28 references that are in the map.
- First posted on arXiv 11 Mar 2026.

## Placed in

- Deep reinforcement learning methods / [RL for robot locomotion and manipulation](https://mlsanity.com/deeprl/rl-for-robot-locomotion-and-manipulation/)

## Builds on (most cited first, 20 of 20)

- [Soft Actor-Critic: Off-Policy Maximum Entropy Deep Reinforcement Learning with a Stochastic Actor](https://mlsanity.com/paper/1801.01290/) (2018)
- [Soft Actor-Critic Algorithms and Applications](https://mlsanity.com/paper/1812.05905/) (2018)
- [Learning Quadrupedal Locomotion over Challenging Terrain](https://mlsanity.com/paper/2010.11251/) (2020)
- [Learning to Walk in Minutes Using Massively Parallel Deep Reinforcement Learning](https://mlsanity.com/paper/2109.11978/) (2021)
- [DayDreamer: World Models for Physical Robot Learning](https://mlsanity.com/paper/2206.14176/) (2022)
- [Legged Locomotion in Challenging Terrains using Egocentric Vision](https://mlsanity.com/paper/2211.07638/) (2022)
- [MuJoCo Playground](https://mlsanity.com/paper/2502.08844/) (2025)
- [RLOC: Terrain-Aware Legged Locomotion using Reinforcement Learning and Optimal Control](https://mlsanity.com/paper/2012.03094/) (2020)
- [DTC: Deep Tracking Control](https://mlsanity.com/paper/2309.15462/) (2023)
- [Fast and Efficient Locomotion via Learned Gait Transitions](https://mlsanity.com/paper/2104.04644/) (2021)
- [Perceptive Locomotion through Nonlinear Model Predictive Control](https://mlsanity.com/paper/2208.08373/) (2022)
- [Optimization-Based Control for Dynamic Legged Robots](https://mlsanity.com/paper/2211.11644/) (2022)
- [RL + Model-based Control: Using On-demand Optimal Control to Learn Versatile Legged Locomotion](https://mlsanity.com/paper/2305.17842/) (2023)
- [Whole-Body Nonlinear Model Predictive Control Through Contacts for Quadrupeds](https://mlsanity.com/paper/1712.02889/) (2017)
- [Whole-Body MPC and Online Gait Sequence Generation for Wheeled-Legged Robots](https://mlsanity.com/paper/2010.06322/) (2020)
- [Rolling in the Deep -- Hybrid Locomotion for Wheeled-Legged Robots using Online Trajectory Optimization](https://mlsanity.com/paper/1909.07193/) (2019)
- [Contact-Implicit Model Predictive Control: Controlling Diverse Quadruped Motions Without Pre-Planned Contact Modes or Trajectories](https://mlsanity.com/paper/2312.08961/) (2023)
- [CusADi: A GPU Parallelization Framework for Symbolic Expressions and Optimal Control](https://mlsanity.com/paper/2408.09662/) (2024)
- [Inverse-Dynamics MPC via Nullspace Resolution](https://mlsanity.com/paper/2209.05375/) (2022)
- [Non-Gaited Legged Locomotion with Monte-Carlo Tree Search and Supervised Learning](https://mlsanity.com/paper/2408.07508/) (2024)

## Abstract (arXiv)

We propose a contact-explicit hierarchical architecture coupling Reinforcement Learning (RL) and Model Predictive Control (MPC), where a high-level RL agent provides gait and navigation commands to a low-level locomotion MPC. This offloads the combinatorial burden of contact timing from the MPC by learning acyclic gaits through trial and error in simulation. We show that only a minimal set of rewards and limited tuning are required to obtain effective policies. We validate the architecture in simulation across robotic platforms spanning 50 kg to 120 kg and different MPC implementations, observing the emergence of acyclic gaits and timing adaptations in flat-terrain legged and hybrid locomotion, and further demonstrating extensibility to non-flat terrains. Across all platforms, we achieve zero-shot sim-to-sim transfer without domain randomization, and we further demonstrate zero-shot sim-to-real transfer without domain randomization on Centauro, our 120 kg wheeled-legged humanoid robot. We make our software framework and evaluation results publicly available at https://github.com/AndrePatri/AugMPC.