# Mechanical Intelligence-Aware Curriculum Reinforcement Learning for Humanoids with Parallel Actuation

arXiv: https://arxiv.org/abs/2507.00273 · HTML: https://mlsanity.com/paper/2507.00273/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in RL for robot locomotion and manipulation, one of 15 directions in Deep reinforcement learning methods.
- Builds on 21 of its 39 references that are in the map.
- First posted on arXiv 30 Jun 2025.

## Placed in

- Deep reinforcement learning methods / [RL for robot locomotion and manipulation](https://mlsanity.com/deeprl/rl-for-robot-locomotion-and-manipulation/)

## Builds on (most cited first, 21 of 21)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Trust Region Policy Optimization](https://mlsanity.com/paper/1502.05477/) (2015)
- [Learning Quadrupedal Locomotion over Challenging Terrain](https://mlsanity.com/paper/2010.11251/) (2020)
- [Learning to Walk in Minutes Using Massively Parallel Deep Reinforcement Learning](https://mlsanity.com/paper/2109.11978/) (2021)
- [Sim-to-Real Transfer in Deep Reinforcement Learning for Robotics: a Survey](https://mlsanity.com/paper/2009.13303/) (2020)
- [Curriculum Learning for Reinforcement Learning Domains: A Framework and Survey](https://mlsanity.com/paper/2003.04960/) (2020)
- [Reverse Curriculum Generation for Reinforcement Learning](https://mlsanity.com/paper/1707.05300/) (2017)
- [Real-World Humanoid Locomotion with Reinforcement Learning](https://mlsanity.com/paper/2303.03381/) (2023)
- [Learning Agile Soccer Skills for a Bipedal Robot with Deep Reinforcement Learning](https://mlsanity.com/paper/2304.13653/) (2023)
- [Safe Reinforcement Learning in Constrained Markov Decision Processes](https://mlsanity.com/paper/2008.06626/) (2020)
- [TRANSIC: Sim-to-Real Policy Transfer by Learning from Online Correction](https://mlsanity.com/paper/2405.10315/) (2024)
- [The MIT Humanoid Robot: Design, Motion Planning, and Control For Acrobatic Behaviors](https://mlsanity.com/paper/2104.09025/) (2021)
- [Crossing The Gap: A Deep Dive into Zero-Shot Sim-to-Real Transfer for Dynamics](https://mlsanity.com/paper/2008.06686/) (2020)
- [Continual Domain Randomization](https://mlsanity.com/paper/2403.12193/) (2024)
- [SCALER: A Tough Versatile Quadruped Free-Climber Robot](https://mlsanity.com/paper/2207.01180/) (2022)
- [BayRnTune: Adaptive Bayesian Domain Randomization via Strategic Fine-tuning](https://mlsanity.com/paper/2310.10606/) (2023)
- [Robust Humanoid Walking on Compliant and Uneven Terrain with Deep Reinforcement Learning](https://mlsanity.com/paper/2504.13619/) (2025)
- [Design Optimization of Wire Arrangement with Variable Relay Points in Numerical Simulation for Tendon-driven Robots](https://mlsanity.com/paper/2401.02730/) (2024)
- [URDF+: An Enhanced URDF for Robots with Kinematic Loops](https://mlsanity.com/paper/2411.19753/) (2024)
- [SCALER: Versatile Multi-Limbed Robot for Free-Climbing in Extreme Terrains](https://mlsanity.com/paper/2312.04856/) (2023)
- [Optimal Control of Walkers with Parallel Actuation](https://mlsanity.com/paper/2504.00642/) (2025)

## Built on by (most cited first, 1 of 1)

- [NaviGait: Navigating Dynamically Feasible Gait Libraries using Deep Reinforcement Learning](https://mlsanity.com/paper/2510.11542/) (2025)

## Abstract (arXiv)

Reinforcement learning (RL) has enabled advances in humanoid robot locomotion, yet most learning frameworks do not account for mechanical intelligence embedded in parallel actuation mechanisms due to limitations in simulator support for closed kinematic chains. This omission can lead to inaccurate motion modeling and suboptimal policies, particularly for robots with high actuation complexity. This paper presents general formulations and simulation methods for three types of parallel mechanisms: a differential pulley, a five-bar linkage, and a four-bar linkage, and trains a parallel-mechanism aware policy through an end-to-end curriculum RL framework for BRUCE, a kid-sized humanoid robot. Unlike prior approaches that rely on simplified serial approximations, we simulate all closed-chain constraints natively using GPU-accelerated MuJoCo (MJX), preserving the hardware's mechanical nonlinear properties during training. We benchmark our RL approach against a model predictive controller (MPC), demonstrating better surface generalization and performance in real-world zero-shot deployment. This work highlights the computational approaches and performance benefits of fully simulating parallel mechanisms in end-to-end learning pipelines for legged humanoids. Project codes with parallel mechanisms: https://github.com/alvister88/og_bruce