# HIL: Hybrid Imitation Learning of Diverse Parkour Skills from Videos

arXiv: https://arxiv.org/abs/2505.12619 · HTML: https://mlsanity.com/paper/2505.12619/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 19 May 2025.

## Built on by (most cited first, 6 of 6)

- [KungfuBot2: Learning Versatile Motion Skills for Humanoid Whole-Body Control](https://mlsanity.com/paper/2509.16638/) (2025)
- [InterPrior: Scaling Generative Control for Physics-Based Human-Object Interactions](https://mlsanity.com/paper/2602.06035/) (2026)
- [InterAgent: Physics-based Multi-agent Command Execution via Diffusion on Interaction Graphs](https://mlsanity.com/paper/2512.07410/) (2025)
- [Learning to Control Physically-simulated 3D Characters via Generating and Mimicking 2D Motions](https://mlsanity.com/paper/2512.08500/) (2025)
- [Generalizing from References using a Multi-Task Reference and Goal-Driven RL Framework](https://mlsanity.com/paper/2602.20375/) (2026)
- [CRISP: Contact-Guided Real2Sim from Monocular Video with Planar Scene Primitives](https://mlsanity.com/paper/2512.14696/) (2025)

## Abstract (arXiv)

Data-driven methods leveraging deep reinforcement learning have become the dominant paradigm for developing controllers that enable physically simulated characters to produce natural human-like behaviors. However, these data-driven methods often struggle to adapt to novel environments and compose diverse skills to perform more complex interaction tasks with the environment. To address these challenges, we propose a hybrid imitation learning (HIL) framework that combines motion tracking, for precise skill replication, with adversarial imitation learning, to enhance adaptability and skill composition, enabling robust dynamic control for highly athletic behaviors. This hybrid learning framework is implemented through parallel multi-task environments and a unified observation space, utilizing a goal-conditioned representation to facilitate knowledge-sharing across the hybrid parallel environments. We demonstrate the effectiveness of HIL on a parkour-style obstacle traversal task and a heading control task. Our framework enables a unified controller that not only preserves the naturalness of reference motion data, but also generalizes effectively to challenging new environments. Evaluations across procedurally generated tasks and baselines show that our method improves motion quality, increases skill diversity, and achieves competitive task completion compared to previous learning-based approaches. Results are best visualized through https://jiashunwang.github.io/HIL