mlsanityData to 14 Oct 2026

MoCaNet: Motion Retargeting in-the-wild via Canonicalization Networks

arXiv 2112.10082 ↗First posted 19 Dec 2021cs.CVcited by 13

Not on the map. Why?

Abstract (arXiv)

We present a novel framework that brings the 3D motion retargeting task from controlled environments to in-the-wild scenarios. In particular, our method is capable of retargeting body motion from a character in a 2D monocular video to a 3D character without using any motion capture system or 3D reconstruction procedure. It is designed to leverage massive online videos for unsupervised training, needless of 3D annotations or motion-body pairing information. The proposed method is built upon two novel canonicalization operations, structure canonicalization and view canonicalization. Trained with the canonicalization operations and the derived regularizations, our method learns to factorize a skeleton sequence into three independent semantic subspaces, i.e., motion, structure, and view angle. The disentangled representation enables motion retargeting from 2D to 3D with high precision. Our method achieves superior performance on motion transfer benchmarks with large body variations and challenging actions. Notably, the canonicalized skeleton sequence could serve as a disentangled and interpretable representation of human motion that benefits action analysis and motion retrieval.

Key facts. Cited by 13 papers in the map (data through 14 Oct 2026). Builds on 43 of its 69 references that are in the map. First posted on arXiv 19 Dec 2021.

Builds on

43 of 69 references.

References · 43

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  3. 2018On the Continuity of Rotation Representations in Neural Networkscited by 1,214
  4. 2018OpenPose: Realtime Multi-Person 2D Pose Estimation using Part Affinity Fieldscited by 1,162
  5. 2016InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Netscited by 860
  6. 2018Multimodal Unsupervised Image-to-Image Translationcited by 795
  7. 2019VIBE: Video Inference for Human Body Pose and Shape Estimationcited by 620
  8. 2018DensePose: Dense Human Pose Estimation In The Wildcited by 575
  9. 2017MoCoGAN: Decomposing Motion and Content for Video Generationcited by 524
  10. 2020PIFuHD: Multi-Level Pixel-Aligned Implicit Function for High-Resolution 3D Human Digitizationcited by 521
  11. 2014Semi-Supervised Learning with Deep Generative Modelscited by 496
  12. 20183D human pose estimation in video with temporal convolutions and semi-supervised trainingcited by 403
31 more
  1. 2018Video-to-Video Synthesiscited by 390
  2. 2018Everybody Dance Nowcited by 315
  3. 2018Diverse Image-to-Image Translation via Disentangled Representationscited by 296
  4. 2021Revisiting Skeleton-based Action Recognitioncited by 257
  5. 2020Pose2Mesh: Graph Convolutional Network for 3D Human Pose and Mesh Recovery from a 2D Human Posecited by 248
  6. 2018Video Based Reconstruction of 3D People Modelscited by 238
  7. 2019Few-shot Video-to-Video Synthesiscited by 197
  8. 2017Decomposing Motion and Content for Natural Video Sequence Predictioncited by 169
  9. 2019Liquid Warping GAN: A Unified Framework for Human Motion Imitation, Appearance Transfer and Novel View Synthesiscited by 161
  10. 2019NASA: Neural Articulated Shape Approximationcited by 149
  11. 2017Unsupervised Learning of Disentangled Representations from Videocited by 139
  12. 2019Camera Distance-aware Top-down Approach for 3D Multi-person Pose Estimation from a Single RGB Imagecited by 132
  13. 2020Skeleton-Aware Networks for Deep Motion Retargetingcited by 124
  14. 2019Unsupervised 3D Pose Estimation with Geometric Self-Supervisioncited by 95
  15. 2018Neural Kinematic Networks for Unsupervised Motion Retargettingcited by 88
  16. 2016Disentangling factors of variation in deep representations using adversarial trainingcited by 87
  17. 20183D Human Pose Estimation in the Wild by Adversarial Learningcited by 86
  18. 2020CanonPose: Self-Supervised Monocular 3D Human Pose Estimation in the Wildcited by 65
  19. 2019C3DPO: Canonical 3D Pose Networks for Non-Rigid Structure From Motioncited by 61
  20. 2018Photo Wake-Up: 3D Character Animation from a Single Photocited by 60
  21. 2020UniPose: Unified Human Pose Estimation in Single Images and Videoscited by 49
  22. 2019Learning Character-Agnostic Motion for Motion Retargeting in 2Dcited by 45
  23. 2021Locally Aware Piecewise Transformation Fields for 3D Human Mesh Registrationcited by 41
  24. 2020TransMoMo: Invariance-Driven Unsupervised Video Motion Retargetingcited by 36
  25. 2020Motion Capture from Internet Videoscited by 33
  26. 2018Deep Video-Based Performance Cloningcited by 32
  27. 2020Unsupervised 3D Human Pose Representation with Viewpoint and Pose Disentanglementcited by 32
  28. 2019Weakly-Supervised Discovery of Geometry-Aware Representation for 3D Human Pose Estimationcited by 23
  29. 2020Toward fast and accurate human pose estimation via soft-gated skip connectionscited by 22
  30. 2020Learning View-Disentangled Human Pose Representation by Contrastive Cross-View Mutual Information Maximizationcited by 19
  31. 2019C-3PO: Cyclic-Three-Phase Optimization for Human-Robot Motion Retargeting based on Reinforcement Learningcited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 3; 2023: 3; 2024: 3; 2025: 1; 2026: 3.

13 papers.

Sort:

By area:

  1. 2022MotionBERT: A Unified Perspective on Learning Human Motion Representationscited by 193
  2. 2023UnifiedGesture: A Unified Gesture Synthesis Model for Multiple Skeletonscited by 15
  3. 2025AthletePose3D: A Benchmark Dataset for 3D Human Pose Estimation and Kinematic Validation in Athletic Movementscited by 14
  4. 2024Deep learning for 3D human pose estimation and mesh recovery: A surveycited by 11
  5. 2023HMC: Hierarchical Mesh Coarsening for Skeleton-free Motion Retargetingcited by 6
  6. 2023Real-time Animation Generation and Control on Rigged Models via Large Language Modelscited by 5
  7. 2024Motion Keyframe Interpolation for Any Human Skeleton via Temporally Consistent Point Cloud Sampling and Reconstructioncited by 5
  8. 2022An Identity-Preserved Framework for Human Motion Transfercited by 2
  9. 2022Neural Sign Reenactor: Deep Photorealistic Sign Language Retargetingcited by 2
  10. 2026ReActor: Reinforcement Learning for Physics-Aware Motion Retargetingcited by 2
  11. 2024Redefining Data Pairing for Motion Retargeting Leveraging a Human Body Priorcited by 0
  12. 2026AnyAct: Towards Human Reenactment of Character Motion From Videocited by 0
1 more
  1. 2026ReFM: Semantic-Aware Refinement Flow Model for Motion Retargetingcited by 0