# I-CTRL: Imitation to Control Humanoid Robots Through Constrained Reinforcement Learning

arXiv: https://arxiv.org/abs/2405.08726 · HTML: https://mlsanity.com/paper/2405.08726/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Placed in RL for robot locomotion and manipulation, one of 15 directions in Deep reinforcement learning methods.
- Builds on 22 of its 41 references that are in the map.
- First posted on arXiv 14 May 2024.

## Placed in

- Deep reinforcement learning methods / [RL for robot locomotion and manipulation](https://mlsanity.com/deeprl/rl-for-robot-locomotion-and-manipulation/)

## Builds on (most cited first, 22 of 22)

- [Proximal Policy Optimization Algorithms](https://mlsanity.com/paper/1707.06347/) (2017)
- [Playing Atari with Deep Reinforcement Learning](https://mlsanity.com/paper/1312.5602/) (2013)
- [Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs)](https://mlsanity.com/paper/1511.07289/) (2015)
- [Benchmarking Deep Reinforcement Learning for Continuous Control](https://mlsanity.com/paper/1604.06778/) (2016)
- [Learning Hand-Eye Coordination for Robotic Grasping with Deep Learning and Large-Scale Data Collection](https://mlsanity.com/paper/1603.02199/) (2016)
- [Multi-Goal Reinforcement Learning: Challenging Robotics Environments and Request for Research](https://mlsanity.com/paper/1802.09464/) (2018)
- [Perpetual Humanoid Control for Real-time Simulated Avatars](https://mlsanity.com/paper/2305.06456/) (2023)
- [Emergence of Locomotion Behaviours in Rich Environments](https://mlsanity.com/paper/1707.02286/) (2017)
- [Legged Locomotion in Challenging Terrains using Egocentric Vision](https://mlsanity.com/paper/2211.07638/) (2022)
- [Skeleton-Aware Networks for Deep Motion Retargeting](https://mlsanity.com/paper/2005.05732/) (2020)
- [Deep Whole-Body Control: Learning a Unified Policy for Manipulation and Locomotion](https://mlsanity.com/paper/2210.10044/) (2022)
- [Reinforcement Learning for Robust Parameterized Locomotion Control of Bipedal Robots](https://mlsanity.com/paper/2103.14295/) (2021)
- [Dynamics-Regulated Kinematic Policy for Egocentric Pose Estimation](https://mlsanity.com/paper/2106.05969/) (2021)
- [Neural Kinematic Networks for Unsupervised Motion Retargetting](https://mlsanity.com/paper/1804.05653/) (2018)
- [Ego-Pose Estimation and Forecasting as Real-Time PD Control](https://mlsanity.com/paper/1906.03173/) (2019)
- [Skinned Motion Retargeting with Residual Perception of Motion Semantics & Geometry](https://mlsanity.com/paper/2303.08658/) (2023)
- [Contact-Aware Retargeting of Skinned Motion](https://mlsanity.com/paper/2109.07431/) (2021)
- [Guided Constrained Policy Optimization for Dynamic Quadrupedal Robot Locomotion](https://mlsanity.com/paper/2002.09676/) (2020)
- [Whole-Body Geometric Retargeting for Humanoid Robots](https://mlsanity.com/paper/1909.10080/) (2019)
- [SLoMo: A General System for Legged Robot Motion Imitation from Casual Videos](https://mlsanity.com/paper/2304.14389/) (2023)
- [ImitationNet: Unsupervised Human-to-Robot Motion Retargeting via Shared Latent Space](https://mlsanity.com/paper/2309.05310/) (2023)
- [Walking Control Based on Step Timing Adaptation](https://mlsanity.com/paper/1704.01271/) (2017)

## Built on by (most cited first, 5 of 5)

- [RobotDancing: Residual-Action Reinforcement Learning Enables Robust Long-Horizon Humanoid Motion Tracking](https://mlsanity.com/paper/2509.20717/) (2025)
- [HumanTracker: Towards Comprehensive and Human-Aligned Motion Tracking Benchmark](https://mlsanity.com/paper/2608.13555/) (2026)
- [Know your limits! Optimize the robot's behavior through self-awareness](https://mlsanity.com/paper/2409.10308/) (2024)
- [PTRL: Prior Transfer Deep Reinforcement Learning for Legged Robots Locomotion](https://mlsanity.com/paper/2504.05629/) (2025)
- [From Motion to Behavior: Hierarchical Modeling of Humanoid Generative Behavior Control](https://mlsanity.com/paper/2506.00043/) (2025)

## Abstract (arXiv)

Humanoid robots have the potential to mimic human motions with high visual fidelity, yet translating these motions into practical, physical execution remains a significant challenge. Existing techniques in the graphics community often prioritize visual fidelity over physics-based feasibility, posing a significant challenge for deploying bipedal systems in practical applications. This paper addresses these issues through bounded residual reinforcement learning to produce physics-based high-quality motion imitation onto legged humanoid robots that enhance motion resemblance while successfully following the reference human trajectory. Our framework, Imitation to Control Humanoid Robots Through Bounded Residual Reinforcement Learning (I-CTRL), reformulates motion imitation as a constrained refinement over non-physics-based retargeted motions. I-CTRL excels in motion imitation with simple and unique rewards that generalize across five robots. Moreover, our framework introduces an automatic priority scheduler to manage large-scale motion datasets when efficiently training a unified RL policy across diverse motions. The proposed approach signifies a crucial step forward in advancing the control of bipedal robots, emphasizing the importance of aligning visual and physical realism for successful motion imitation.