# ModSkill: Physical Character Skill Modularization

arXiv: https://arxiv.org/abs/2502.14140 · HTML: https://mlsanity.com/paper/2502.14140/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Placed in Robot manipulation imitation learning, one of 13 directions in Offline RL and imitation learning from data.
- Builds on 45 of its 72 references that are in the map.
- First posted on arXiv 19 Feb 2025.

## Placed in

- Offline RL and imitation learning from data / [Robot manipulation imitation learning](https://mlsanity.com/offlinerl/robot-manipulation-imitation-learning/)

## Builds on (most cited first, 30 of 45)

- [AMASS: Archive of Motion Capture as Surface Shapes](https://mlsanity.com/paper/1904.03278/) (2019)
- [Human Motion Diffusion Model](https://mlsanity.com/paper/2209.14916/) (2022)
- [Isaac Gym: High Performance GPU-Based Physics Simulation For Robot Learning](https://mlsanity.com/paper/2108.10470/) (2021)
- [T2M-GPT: Generating Human Motion from Textual Descriptions with Discrete Representations](https://mlsanity.com/paper/2301.06052/) (2023)
- [Executing your Commands via Motion Diffusion in Latent Space](https://mlsanity.com/paper/2212.04048/) (2022)
- [MotionGPT: Human Motion as a Foreign Language](https://mlsanity.com/paper/2306.14795/) (2023)
- [BEDLAM: A Synthetic Dataset of Bodies Exhibiting Detailed Lifelike Animated Motion](https://mlsanity.com/paper/2306.16940/) (2023)
- [Perpetual Humanoid Control for Real-time Simulated Avatars](https://mlsanity.com/paper/2305.06456/) (2023)
- [Stochastic Scene-Aware Motion Prediction](https://mlsanity.com/paper/2108.08284/) (2021)
- [OmniH2O: Universal and Dexterous Human-to-Humanoid Whole-Body Teleoperation and Learning](https://mlsanity.com/paper/2406.08858/) (2024)
- [Character Controllers Using Motion VAEs](https://mlsanity.com/paper/2103.14274/) (2021)
- [Learning Human-to-Humanoid Real-Time Whole-Body Teleoperation](https://mlsanity.com/paper/2403.04436/) (2024)
- [Universal Humanoid Motion Representations for Physics-Based Control](https://mlsanity.com/paper/2310.04582/) (2023)
- [MaskedMimic: Unified Physics-Based Character Control Through Masked Motion Inpainting](https://mlsanity.com/paper/2409.14393/) (2024)
- [CLoSD: Closing the Loop between Simulation and Diffusion for multi-task character control](https://mlsanity.com/paper/2410.03441/) (2024)
- [MCP: Learning Composable Hierarchical Control with Multiplicative Compositional Policies](https://mlsanity.com/paper/1905.09808/) (2019)
- [FineMoGen: Fine-Grained Spatio-Temporal Motion Generation and Editing](https://mlsanity.com/paper/2312.15004/) (2023)
- [Synthesizing Physical Character-Scene Interactions](https://mlsanity.com/paper/2302.00883/) (2023)
- [Dynamics-Regulated Kinematic Policy for Egocentric Pose Estimation](https://mlsanity.com/paper/2106.05969/) (2021)
- [Neural probabilistic motor primitives for humanoid control](https://mlsanity.com/paper/1811.11711/) (2018)
- [CALM: Conditional Adversarial Latent Models for Directable Virtual Characters](https://mlsanity.com/paper/2305.02195/) (2023)
- [TLControl: Trajectory and Language Control for Human Motion Synthesis](https://mlsanity.com/paper/2311.17135/) (2023)
- [Towards Diverse and Natural Scene-aware 3D Human Motion Synthesis](https://mlsanity.com/paper/2205.13001/) (2022)
- [EMDM: Efficient Motion Diffusion Model for Fast and High-Quality Motion Generation](https://mlsanity.com/paper/2312.02256/) (2023)
- [QuestSim: Human Motion Tracking from Sparse Sensors with Simulated Avatars](https://mlsanity.com/paper/2209.09391/) (2022)
- [CoMo: Controllable Motion Generation through Language Guided Pose Code Editing](https://mlsanity.com/paper/2403.13900/) (2024)
- [Residual Force Control for Agile Human Behavior Imitation and Extended Motion Synthesis](https://mlsanity.com/paper/2006.07364/) (2020)
- [Motion Puzzle: Arbitrary Motion Style Transfer by Body Part](https://mlsanity.com/paper/2202.05274/) (2022)
- [C·ASE: Learning Conditional Adversarial Skill Embeddings for Physics-based Characters](https://mlsanity.com/paper/2309.11351/) (2023)
- [PADL: Language-Directed Physics-Based Character Control](https://mlsanity.com/paper/2301.13868/) (2023)

## Built on by (most cited first, 7 of 7)

- [Go to Zero: Towards Zero-shot Motion Generation with Million-scale Data](https://mlsanity.com/paper/2507.07095/) (2025)
- [MOSPA: Human Motion Generation Driven by Spatial Audio](https://mlsanity.com/paper/2507.11949/) (2025)
- [Text2Interact: High-Fidelity and Diverse Text-to-Two-Person Interaction Generation](https://mlsanity.com/paper/2510.06504/) (2025)
- [Auto-Connect: Connectivity-Preserving RigFormer with Direct Preference Optimization](https://mlsanity.com/paper/2506.11430/) (2025)
- [EgoReAct: Egocentric Video-Driven 3D Human Reaction Generation](https://mlsanity.com/paper/2512.22808/) (2025)
- [InterAgent: Physics-based Multi-agent Command Execution via Diffusion on Interaction Graphs](https://mlsanity.com/paper/2512.07410/) (2025)
- [TeamHOI: Learning a Unified Policy for Cooperative Human-Object Interactions with Any Team Size](https://mlsanity.com/paper/2603.07988/) (2026)

## Abstract (arXiv)

Human motion is highly diverse and dynamic, posing challenges for imitation learning algorithms that aim to generalize motor skills for controlling simulated characters. Previous methods typically rely on a universal full-body controller for tracking reference motion (tracking-based model) or a unified full-body skill embedding space (skill embedding). However, these approaches often struggle to generalize and scale to larger motion datasets. In this work, we introduce a novel skill learning framework, ModSkill, that decouples complex full-body skills into compositional, modular skills for independent body parts. Our framework features a skill modularization attention layer that processes policy observations into modular skill embeddings that guide low-level controllers for each body part. We also propose an Active Skill Learning approach with Generative Adaptive Sampling, using large motion generation models to adaptively enhance policy learning in challenging tracking scenarios. Our results show that this modularized skill learning framework, enhanced by generative sampling, outperforms existing methods in precise full-body motion tracking and enables reusable skill embeddings for diverse goal-driven tasks.