# Learning Character-Agnostic Motion for Motion Retargeting in 2D

arXiv: https://arxiv.org/abs/1905.01680 · HTML: https://mlsanity.com/paper/1905.01680/

## Key facts

- Cited by 45 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 5 May 2019.

## Built on by (most cited first, 30 of 45)

- [DexMV: Imitation Learning for Dexterous Manipulation from Human Videos](https://mlsanity.com/paper/2108.05877/) (2021)
- [Audio-Driven Emotional Video Portraits](https://mlsanity.com/paper/2104.07452/) (2021)
- [Single Motion Diffusion](https://mlsanity.com/paper/2302.05905/) (2023)
- [MoDi: Unconditional Motion Synthesis from Diverse Data](https://mlsanity.com/paper/2206.08010/) (2022)
- [TEDi: Temporally-Entangled Diffusion for Long-Term Motion Synthesis](https://mlsanity.com/paper/2307.15042/) (2023)
- [GeneOH Diffusion: Towards Generalizable Hand-Object Interaction Denoising via Denoising Diffusion](https://mlsanity.com/paper/2402.14810/) (2024)
- [Contact-Aware Retargeting of Skinned Motion](https://mlsanity.com/paper/2109.07431/) (2021)
- [Pretrained Diffusion Models for Unified Human Motion Synthesis](https://mlsanity.com/paper/2212.02837/) (2022)
- [GANimator: Neural Motion Synthesis from a Single Sequence](https://mlsanity.com/paper/2205.02625/) (2022)
- [ViA: View-invariant Skeleton Action Representation Learning via Motion Retargeting](https://mlsanity.com/paper/2209.00065/) (2022)
- [Deep Learning for Visual Speech Analysis: A Survey](https://mlsanity.com/paper/2205.10839/) (2022)
- [HUMOS: Human Motion Model Conditioned on Body Shape](https://mlsanity.com/paper/2409.03944/) (2024)
- [Video-driven Neural Physically-based Facial Asset for Production](https://mlsanity.com/paper/2202.05592/) (2022)
- [Motion Inbetweening via Deep Delta-Interpolator](https://mlsanity.com/paper/2201.06701/) (2022)
- [LAC: Latent Action Composition for Skeleton-based Action Segmentation](https://mlsanity.com/paper/2308.14500/) (2023)
- [MoCaNet: Motion Retargeting in-the-wild via Canonicalization Networks](https://mlsanity.com/paper/2112.10082/) (2021)
- [Real-time RGBD-based Extended Body Pose Estimation](https://mlsanity.com/paper/2103.03663/) (2021)
- [ChatHuman: Chatting about 3D Humans with Tools](https://mlsanity.com/paper/2405.04533/) (2024)
- [Shape My Moves: Text-Driven Shape-Aware Synthesis of Human Motions](https://mlsanity.com/paper/2504.03639/) (2025)
- [Pose-aware Attention Network for Flexible Motion Retargeting by Body Part](https://mlsanity.com/paper/2306.08006/) (2023)
- [Conditional Temporal Variational AutoEncoder for Action Video Prediction](https://mlsanity.com/paper/2108.05658/) (2021)
- [A Shape-Aware Retargeting Approach to Transfer Human Motion and Appearance in Monocular Videos](https://mlsanity.com/paper/2103.15596/) (2021)
- [Correspondence-free online human motion retargeting](https://mlsanity.com/paper/2302.00556/) (2023)
- [MotionRAG: Motion Retrieval-Augmented Image-to-Video Generation](https://mlsanity.com/paper/2509.26391/) (2025)
- [DisUnknown: Distilling Unknown Factors for Disentanglement Learning](https://mlsanity.com/paper/2109.08090/) (2021)
- [Motion Keyframe Interpolation for Any Human Skeleton via Temporally Consistent Point Cloud Sampling and Reconstruction](https://mlsanity.com/paper/2405.07444/) (2024)
- [AnaMoDiff: 2D Analogical Motion Diffusion via Disentangled Denoising](https://mlsanity.com/paper/2402.03549/) (2024)
- [Video Reenactment as Inductive Bias for Content-Motion Disentanglement](https://mlsanity.com/paper/2102.00324/) (2021)
- [JOKR: Joint Keypoint Representation for Unsupervised Cross-Domain Motion Retargeting](https://mlsanity.com/paper/2106.09679/) (2021)
- [Spatial and Surface Correspondence Field for Interaction Transfer](https://mlsanity.com/paper/2405.03221/) (2024)

## Abstract (arXiv)

Analyzing human motion is a challenging task with a wide variety of applications in computer vision and in graphics. One such application, of particular importance in computer animation, is the retargeting of motion from one performer to another. While humans move in three dimensions, the vast majority of human motions are captured using video, requiring 2D-to-3D pose and camera recovery, before existing retargeting approaches may be applied. In this paper, we present a new method for retargeting video-captured motion between different human performers, without the need to explicitly reconstruct 3D poses and/or camera parameters. In order to achieve our goal, we learn to extract, directly from a video, a high-level latent motion representation, which is invariant to the skeleton geometry and the camera view. Our key idea is to train a deep neural network to decompose temporal sequences of 2D poses into three components: motion, skeleton, and camera view-angle. Having extracted such a representation, we are able to re-combine motion with novel skeletons and camera views, and decode a retargeted temporal sequence, which we compare to a ground truth from a synthetic dataset. We demonstrate that our framework can be used to robustly extract human motion from videos, bypassing 3D reconstruction, and outperforming existing retargeting methods, when applied to videos in-the-wild. It also enables additional applications, such as performance cloning, video-driven cartoons, and motion retrieval.