# Reinforcement Learning for Reduced-order Models of Legged Robots

arXiv: https://arxiv.org/abs/2310.09873 · HTML: https://mlsanity.com/paper/2310.09873/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Oct 2023.

## Built on by (most cited first, 5 of 5)

- [RL-augmented MPC Framework for Agile and Robust Bipedal Footstep Locomotion Planning and Control](https://mlsanity.com/paper/2407.17683/) (2024)
- [Automated Hybrid Reward Scheduling via Large Language Models for Robotic Skill Learning](https://mlsanity.com/paper/2505.02483/) (2025)
- [Transfer Learning for a Class of Cascade Dynamical Systems](https://mlsanity.com/paper/2410.06828/) (2024)
- [GPO: Growing Policy Optimization for Legged Robot Locomotion and Whole-Body Control](https://mlsanity.com/paper/2601.20668/) (2026)
- [Safety Guarantees in Zero-Shot Reinforcement Learning for Cascade Dynamical Systems](https://mlsanity.com/paper/2604.10429/) (2026)

## Abstract (arXiv)

Model-based approaches for planning and control for bipedal locomotion have a long history of success. It can provide stability and safety guarantees while being effective in accomplishing many locomotion tasks. Model-free reinforcement learning, on the other hand, has gained much popularity in recent years due to computational advancements. It can achieve high performance in specific tasks, but it lacks physical interpretability and flexibility in re-purposing the policy for a different set of tasks. For instance, we can initially train a neural network (NN) policy using velocity commands as inputs. However, to handle new task commands like desired hand or footstep locations at a desired walking velocity, we must retrain a new NN policy. In this work, we attempt to bridge the gap between these two bodies of work on a bipedal platform. We formulate a model-based reinforcement learning problem to learn a reduced-order model (ROM) within a model predictive control (MPC). Results show a 49% improvement in viable task region size and a 21% reduction in motor torque cost. All videos and code are available at https://sites.google.com/view/ymchen/research/rl-for-roms.