# Constrained Reinforcement Learning for Unstable Point-Feet Bipedal Locomotion Applied to the Bolt Robot

arXiv: https://arxiv.org/abs/2508.02194 · HTML: https://mlsanity.com/paper/2508.02194/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in RL for robot locomotion and manipulation, one of 15 directions in Deep reinforcement learning methods.
- Builds on 32 of its 53 references that are in the map.
- First posted on arXiv 4 Aug 2025.

## Placed in

- Deep reinforcement learning methods / [RL for robot locomotion and manipulation](https://mlsanity.com/deeprl/rl-for-robot-locomotion-and-manipulation/)

## Builds on (most cited first, 30 of 32)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Learning to Walk in Minutes Using Massively Parallel Deep Reinforcement Learning](https://mlsanity.com/paper/2109.11978/) (2021)
- [RMA: Rapid Motor Adaptation for Legged Robots](https://mlsanity.com/paper/2107.04034/) (2021)
- [Orbit: A Unified Simulation Framework for Interactive Robot Learning Environments](https://mlsanity.com/paper/2301.04195/) (2023)
- [Real-World Humanoid Locomotion with Reinforcement Learning](https://mlsanity.com/paper/2303.03381/) (2023)
- [Extreme Parkour with Legged Robots](https://mlsanity.com/paper/2309.14341/) (2023)
- [ASAP: Aligning Simulation and Real-World Physics for Learning Agile Humanoid Whole-Body Skills](https://mlsanity.com/paper/2502.01143/) (2025)
- [Reinforcement Learning for Versatile, Dynamic, and Robust Bipedal Locomotion Control](https://mlsanity.com/paper/2401.16889/) (2024)
- [Concurrent Training of a Control Policy and a State Estimator for Dynamic and Robust Legged Locomotion](https://mlsanity.com/paper/2202.05481/) (2022)
- [Not Only Rewards But Also Constraints: Applications on Legged Robot Locomotion](https://mlsanity.com/paper/2308.12517/) (2023)
- [Optimization-Based Control for Dynamic Legged Robots](https://mlsanity.com/paper/2211.11644/) (2022)
- [Crocoddyl: An Efficient and Versatile Framework for Multi-Contact Optimal Control](https://mlsanity.com/paper/1909.04947/) (2019)
- [CTS: Concurrent Teacher-Student Reinforcement Learning for Legged Locomotion](https://mlsanity.com/paper/2405.10830/) (2024)
- [CaT: Constraints as Terminations for Legged Locomotion Reinforcement Learning](https://mlsanity.com/paper/2403.18765/) (2024)
- [An Open Torque-Controlled Modular Robot Architecture for Legged Locomotion Research](https://mlsanity.com/paper/1910.00093/) (2019)
- [Controlling the Solo12 Quadruped Robot with Deep Reinforcement Learning](https://mlsanity.com/paper/2309.16683/) (2023)
- [Learning Agile Bipedal Motions on a Quadrupedal Robot](https://mlsanity.com/paper/2311.05818/) (2023)
- [SoloParkour: Constrained Reinforcement Learning for Visual Locomotion from Privileged Experience](https://mlsanity.com/paper/2409.13678/) (2024)
- [Bridging the Sim-to-Real Gap for Athletic Loco-Manipulation](https://mlsanity.com/paper/2502.10894/) (2025)
- [Using Deep Reinforcement Learning to Learn High-Level Policies on the ATRIAS Biped](https://mlsanity.com/paper/1809.10811/) (2018)
- [Imitating and Finetuning Model Predictive Control for Robust and Symmetric Quadrupedal Locomotion](https://mlsanity.com/paper/2311.02304/) (2023)
- [Cafe-Mpc: A Cascaded-Fidelity Model Predictive Control Framework with Tuning-Free Whole-Body Control](https://mlsanity.com/paper/2403.03995/) (2024)
- [Reinforcement Learning for Blind Stair Climbing with Legged and Wheeled-Legged Robots](https://mlsanity.com/paper/2402.06143/) (2024)
- [Zero Dynamics, Pendulum Models, and Angular Momentum in Feedback Control of Bipedal Locomotion](https://mlsanity.com/paper/2105.08170/) (2021)
- [Walking Control Based on Step Timing Adaptation](https://mlsanity.com/paper/1704.01271/) (2017)
- [On the Emergence of Whole-body Strategies from Humanoid Robot Push-recovery Learning](https://mlsanity.com/paper/2104.14534/) (2021)
- [A unified framework for walking and running of bipedal robots](https://mlsanity.com/paper/2110.09172/) (2021)
- [A Real-World Quadrupedal Locomotion Benchmark for Offline Reinforcement Learning](https://mlsanity.com/paper/2309.16718/) (2023)
- [Stoch BiRo: Design and Control of a low cost bipedal robot](https://mlsanity.com/paper/2312.06512/) (2023)
- [Whole-body MPC and sensitivity analysis of a real time foot step sequencer for a biped robot Bolt](https://mlsanity.com/paper/2412.01713/) (2024)

## Built on by (most cited first, 1 of 1)

- [Direct Dynamic Retargeting for Humanoid Imitation Learning from Videos](https://mlsanity.com/paper/2605.23762/) (2026)

## Abstract (arXiv)

Bipedal locomotion is a key challenge in robotics, particularly for robots like Bolt, which have a point-foot design. This study explores the control of such underactuated robots using constrained reinforcement learning, addressing their inherent instability, lack of arms, and limited foot actuation. We present a methodology that leverages Constraints-as-Terminations and domain randomization techniques to enable sim-to-real transfer. Through a series of qualitative and quantitative experiments, we evaluate our approach in terms of balance maintenance, velocity control, and responses to slip and push disturbances. Additionally, we analyze autonomy through metrics like the cost of transport and ground reaction force. Our method advances robust control strategies for point-foot bipedal robots, offering insights into broader locomotion.