# Towards High-Quality 3D Motion Transfer with Realistic Apparel Animation

arXiv: https://arxiv.org/abs/2407.11266 · HTML: https://mlsanity.com/paper/2407.11266/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 38 references that are in the map.
- First posted on arXiv 15 Jul 2024.

## Builds on (most cited first, 26 of 26)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [Dynamic Graph CNN for Learning on Point Clouds](https://mlsanity.com/paper/1801.07829/) (2018)
- [AMASS: Archive of Motion Capture as Surface Shapes](https://mlsanity.com/paper/1904.03278/) (2019)
- [Multi-Garment Net: Learning to Dress 3D People from Images](https://mlsanity.com/paper/1908.06903/) (2019)
- [DreamAvatar: Text-and-Shape Guided 3D Human Avatar Generation via Diffusion Models](https://mlsanity.com/paper/2304.00916/) (2023)
- [TailorNet: Predicting Clothing in 3D as a Function of Human Pose, Shape and Garment Style](https://mlsanity.com/paper/2003.04583/) (2020)
- [Skeleton-Aware Networks for Deep Motion Retargeting](https://mlsanity.com/paper/2005.05732/) (2020)
- [Neural Cages for Detail-Preserving 3D Deformations](https://mlsanity.com/paper/1912.06395/) (2019)
- [Neural Kinematic Networks for Unsupervised Motion Retargetting](https://mlsanity.com/paper/1804.05653/) (2018)
- [Modeling Clothing as a Separate Layer for an Animatable Human Avatar](https://mlsanity.com/paper/2106.14879/) (2021)
- [Learning Skeletal Articulations with Neural Blend Shapes](https://mlsanity.com/paper/2105.02451/) (2021)
- [Predicting Animation Skeletons for 3D Articulated Models via Volumetric Nets](https://mlsanity.com/paper/1908.08506/) (2019)
- [Predicting Loose-Fitting Garment Deformations Using Bone-Driven Motion Networks](https://mlsanity.com/paper/2205.01355/) (2022)
- [Neural Pose Transfer by Spatially Adaptive Instance Normalization](https://mlsanity.com/paper/2003.07254/) (2020)
- [Text and Image Guided 3D Avatar Generation and Manipulation](https://mlsanity.com/paper/2202.06079/) (2022)
- [Skinned Motion Retargeting with Residual Perception of Motion Semantics & Geometry](https://mlsanity.com/paper/2303.08658/) (2023)
- [Skeleton-free Pose Transfer for Stylized 3D Characters](https://mlsanity.com/paper/2208.00790/) (2022)
- [Motion Guided Deep Dynamic 3D Garments](https://mlsanity.com/paper/2209.11449/) (2022)
- [A Deep Emulator for Secondary Motion of 3D Characters](https://mlsanity.com/paper/2103.01261/) (2021)
- [Towards Multi-Layered 3D Garments Animation](https://mlsanity.com/paper/2305.10418/) (2023)
- [Zero-shot Pose Transfer for Unrigged Stylized 3D Characters](https://mlsanity.com/paper/2306.00200/) (2023)
- [Weakly-supervised 3D Pose Transfer with Keypoints](https://mlsanity.com/paper/2307.13459/) (2023)
- [Learning Anchor Transformations for 3D Garment Animation](https://mlsanity.com/paper/2304.00761/) (2023)
- [HMC: Hierarchical Mesh Coarsening for Skeleton-free Motion Retargeting](https://mlsanity.com/paper/2303.10941/) (2023)

## Built on by (most cited first, 5 of 5)

- [Auto-Connect: Connectivity-Preserving RigFormer with Direct Preference Optimization](https://mlsanity.com/paper/2506.11430/) (2025)
- [ASMR: Adaptive Skeleton-Mesh Rigging and Skinning via 2D Generative Prior](https://mlsanity.com/paper/2503.13579/) (2025)
- [Gaussian See, Gaussian Do: Semantic 3D Motion Transfer from Multiview Video](https://mlsanity.com/paper/2511.14848/) (2025)
- [Motion Marionette: Rethinking Rigid Motion Transfer via Prior Guidance](https://mlsanity.com/paper/2511.19909/) (2025)
- [UNIC: Neural Garment Deformation Field for Real-time Clothed Character Animation](https://mlsanity.com/paper/2603.25580/) (2026)

## Abstract (arXiv)

Animating stylized characters to match a reference motion sequence is a highly demanded task in film and gaming industries. Existing methods mostly focus on rigid deformations of characters' body, neglecting local deformations on the apparel driven by physical dynamics. They deform apparel the same way as the body, leading to results with limited details and unrealistic artifacts, e.g. body-apparel penetration. In contrast, we present a novel method aiming for high-quality motion transfer with realistic apparel animation. As existing datasets lack annotations necessary for generating realistic apparel animations, we build a new dataset named MMDMC, which combines stylized characters from the MikuMikuDance community with real-world Motion Capture data. We then propose a data-driven pipeline that learns to disentangle body and apparel deformations via two neural deformation modules. For body parts, we propose a geodesic attention block to effectively incorporate semantic priors into skeletal body deformation to tackle complex body shapes for stylized characters. Since apparel motion can significantly deviate from respective body joints, we propose to model apparel deformation in a non-linear vertex displacement field conditioned on its historic states. Extensive experiments show that our method produces results with superior quality for various types of apparel. Our dataset is released in https://github.com/rongakowang/MMDMC.