# High-Fidelity Neural Human Motion Transfer from Monocular Video

arXiv: https://arxiv.org/abs/2012.10974 · HTML: https://mlsanity.com/paper/2012.10974/

## Key facts

- Cited by 31 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Dec 2020.

## Built on by (most cited first, 30 of 31)

- [Neural Actor: Neural Free-view Synthesis of Human Actors with Pose Control](https://mlsanity.com/paper/2106.02019/) (2021)
- [ParCo: Part-Coordinating Text-to-Motion Synthesis](https://mlsanity.com/paper/2403.18512/) (2024)
- [Style and Pose Control for Image Synthesis of Humans from a Single Monocular View](https://mlsanity.com/paper/2102.11263/) (2021)
- [Dynamic Neural Garments](https://mlsanity.com/paper/2102.11811/) (2021)
- [2D Human Pose Estimation: A Survey](https://mlsanity.com/paper/2204.07370/) (2022)
- [TALK-Act: Enhance Textural-Awareness for 2D Speaking Avatar Reenactment with Diffusion Model](https://mlsanity.com/paper/2410.10696/) (2024)
- [Holoported Characters: Real-time Free-viewpoint Rendering of Humans from Sparse RGB Cameras](https://mlsanity.com/paper/2312.07423/) (2023)
- [Dance In the Wild: Monocular Human Animation with Neural Dynamic Appearance Synthesis](https://mlsanity.com/paper/2111.05916/) (2021)
- [Copy Motion From One to Another: Fake Motion Video Generation](https://mlsanity.com/paper/2205.01373/) (2022)
- [Learning Motion-Dependent Appearance for High-Fidelity Rendering of Dynamic Humans from a Single Camera](https://mlsanity.com/paper/2203.12780/) (2022)
- [Fast-Vid2Vid: Spatial-Temporal Compression for Video-to-Video Synthesis](https://mlsanity.com/paper/2207.05049/) (2022)
- [HDHumans: A Hybrid Approach for High-fidelity Digital Humans](https://mlsanity.com/paper/2210.12003/) (2022)
- [Semantic Human Parsing via Scalable Semantic Transfer over Multiple Label Domains](https://mlsanity.com/paper/2304.04140/) (2023)
- [Conditional Temporal Variational AutoEncoder for Action Video Prediction](https://mlsanity.com/paper/2108.05658/) (2021)
- [Neural Human Deformation Transfer](https://mlsanity.com/paper/2109.01588/) (2021)
- [BiPO: Bidirectional Partial Occlusion Network for Text-to-Motion Synthesis](https://mlsanity.com/paper/2412.00112/) (2024)
- [Motion and Appearance Adaptation for Cross-Domain Motion Transfer](https://mlsanity.com/paper/2209.14529/) (2022)
- [Temporal Consistency-Aware Text-to-Motion Generation](https://mlsanity.com/paper/2602.18057/) (2026)
- [Text-Conditional Contextualized Avatars For Zero-Shot Personalization](https://mlsanity.com/paper/2304.07410/) (2023)
- [FIANCEE: Faster Inference of Adversarial Networks via Conditional Early Exits](https://mlsanity.com/paper/2304.10306/) (2023)
- [An Identity-Preserved Framework for Human Motion Transfer](https://mlsanity.com/paper/2204.06862/) (2022)
- [Bidirectional Temporal Diffusion Model for Temporally Consistent Human Animation](https://mlsanity.com/paper/2307.00574/) (2023)
- [Dynamic Appearance Modeling of Clothed 3D Human Avatars using a Single Camera](https://mlsanity.com/paper/2312.16842/) (2023)
- [Recoverable Anonymization for Pose Estimation: A Privacy-Enhancing Approach](https://mlsanity.com/paper/2409.02715/) (2024)
- [Deep Learning and Synthetic Media](https://mlsanity.com/paper/2205.05764/) (2022)
- [Can Pose Transfer Models Generate Realistic Human Motion?](https://mlsanity.com/paper/2501.15648/) (2025)
- [Multi-granular body modeling with Redundancy-Free Spatiotemporal Fusion for Text-Driven Motion Generation](https://mlsanity.com/paper/2503.06897/) (2025)
- [ParTY: Part-Guidance for Expressive Text-to-Motion Synthesis](https://mlsanity.com/paper/2603.09611/) (2026)
- [AudioEar: Single-View Ear Reconstruction for Personalized Spatial Audio](https://mlsanity.com/paper/2301.12613/) (2023)
- [Garment Animation NeRF with Color Editing](https://mlsanity.com/paper/2407.19774/) (2024)

## Abstract (arXiv)

Video-based human motion transfer creates video animations of humans following a source motion. Current methods show remarkable results for tightly-clad subjects. However, the lack of temporally consistent handling of plausible clothing dynamics, including fine and high-frequency details, significantly limits the attainable visual quality. We address these limitations for the first time in the literature and present a new framework which performs high-fidelity and temporally-consistent human motion transfer with natural pose-dependent non-rigid deformations, for several types of loose garments. In contrast to the previous techniques, we perform image generation in three subsequent stages, synthesizing human shape, structure, and appearance. Given a monocular RGB video of an actor, we train a stack of recurrent deep neural networks that generate these intermediate representations from 2D poses and their temporal derivatives. Splitting the difficult motion transfer problem into subtasks that are aware of the temporal motion context helps us to synthesize results with plausible dynamics and pose-dependent detail. It also allows artistic control of results by manipulation of individual framework stages. In the experimental results, we significantly outperform the state-of-the-art in terms of video realism. Our code and data will be made publicly available.