# Weakly-supervised 3D Pose Transfer with Keypoints

arXiv: https://arxiv.org/abs/2307.13459 · HTML: https://mlsanity.com/paper/2307.13459/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 32 references that are in the map.
- First posted on arXiv 25 Jul 2023.

## Builds on (most cited first, 18 of 18)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [Arbitrary Style Transfer in Real-time with Adaptive Instance Normalization](https://mlsanity.com/paper/1703.06868/) (2017)
- [3D Menagerie: Modeling the 3D shape and pose of animals](https://mlsanity.com/paper/1611.07700/) (2016)
- [HybrIK: A Hybrid Analytical-Neural Inverse Kinematics Solution for 3D Human Pose and Shape Estimation](https://mlsanity.com/paper/2011.14672/) (2020)
- [Neural Cages for Detail-Preserving 3D Deformations](https://mlsanity.com/paper/1912.06395/) (2019)
- [NeuroMorph: Unsupervised Shape Interpolation and Correspondence in One Go](https://mlsanity.com/paper/2106.09431/) (2021)
- [Unsupervised Shape and Pose Disentanglement for 3D Meshes](https://mlsanity.com/paper/2007.11341/) (2020)
- [LIMP: Learning Latent Shape Representations with Metric Preservation Priors](https://mlsanity.com/paper/2003.12283/) (2020)
- [KeypointDeformer: Unsupervised 3D Keypoint Discovery for Shape Control](https://mlsanity.com/paper/2104.11224/) (2021)
- [Neural Pose Transfer by Spatially Adaptive Instance Normalization](https://mlsanity.com/paper/2003.07254/) (2020)
- [DeepMetaHandles: Learning Deformation Meta-Handles of 3D Meshes with Biharmonic Coordinates](https://mlsanity.com/paper/2102.09105/) (2021)
- [3D Pose Transfer with Correspondence Learning and Mesh Refinement](https://mlsanity.com/paper/2109.15025/) (2021)
- [Skeleton-free Pose Transfer for Stylized 3D Characters](https://mlsanity.com/paper/2208.00790/) (2022)
- [Unsupervised 3D Pose Transfer with Cross Consistency and Dual Reconstruction](https://mlsanity.com/paper/2211.10278/) (2022)
- [Intrinsic-Extrinsic Preserved GANs for Unsupervised 3D Pose Transfer](https://mlsanity.com/paper/2108.07520/) (2021)
- [Geometry-Contrastive Transformer for Generalized 3D Pose Transfer](https://mlsanity.com/paper/2112.07374/) (2021)
- [CartoonRenderer: An Instance-based Multi-Style Cartoon Image Translator](https://mlsanity.com/paper/1911.06102/) (2019)

## Built on by (most cited first, 12 of 12)

- [MagicPose4D: Crafting Articulated Models with Appearance and Motion Control](https://mlsanity.com/paper/2405.14017/) (2024)
- [PhysRig: Differentiable Physics-Based Skinning and Rigging Framework for Realistic Articulated Object Modeling](https://mlsanity.com/paper/2506.20936/) (2025)
- [Auto-Connect: Connectivity-Preserving RigFormer with Direct Preference Optimization](https://mlsanity.com/paper/2506.11430/) (2025)
- [Forge4D: Feed-Forward 4D Human Reconstruction and Interpolation from Uncalibrated Sparse-view Videos](https://mlsanity.com/paper/2509.24209/) (2025)
- [Neural Pose Representation Learning for Generating and Transferring Non-Rigid Object Poses](https://mlsanity.com/paper/2406.09728/) (2024)
- [Motion Keyframe Interpolation for Any Human Skeleton via Temporally Consistent Point Cloud Sampling and Reconstruction](https://mlsanity.com/paper/2405.07444/) (2024)
- [Towards High-Quality 3D Motion Transfer with Realistic Apparel Animation](https://mlsanity.com/paper/2407.11266/) (2024)
- [Towards Robust 3D Pose Transfer with Adversarial Learning](https://mlsanity.com/paper/2404.02242/) (2024)
- [Gaussian See, Gaussian Do: Semantic 3D Motion Transfer from Multiview Video](https://mlsanity.com/paper/2511.14848/) (2025)
- [MimiCAT: Mimic with Correspondence-Aware Cascade-Transformer for Category-Free 3D Pose Transfer](https://mlsanity.com/paper/2511.18370/) (2025)
- [MorphGS: Morphology-Adaptive Articulated 3D Motion Transfer from Videos](https://mlsanity.com/paper/2601.02716/) (2026)
- [BLARM: Animating 3D Objects from Video via Blending Latent Rigid Motion Primitives](https://mlsanity.com/paper/2608.31113/) (2026)

## Abstract (arXiv)

The main challenges of 3D pose transfer are: 1) Lack of paired training data with different characters performing the same pose; 2) Disentangling pose and shape information from the target mesh; 3) Difficulty in applying to meshes with different topologies. We thus propose a novel weakly-supervised keypoint-based framework to overcome these difficulties. Specifically, we use a topology-agnostic keypoint detector with inverse kinematics to compute transformations between the source and target meshes. Our method only requires supervision on the keypoints, can be applied to meshes with different topologies and is shape-invariant for the target which allows extraction of pose-only information from the target meshes without transferring shape information. We further design a cycle reconstruction to perform self-supervised pose transfer without the need for ground truth deformed mesh with the same pose and shape as the target and source, respectively. We evaluate our approach on benchmark human and animal datasets, where we achieve superior performance compared to the state-of-the-art unsupervised approaches and even comparable performance with the fully supervised approaches. We test on the more challenging Mixamo dataset to verify our approach's ability in handling meshes with different topologies and complex clothes. Cross-dataset evaluation further shows the strong generalization ability of our approach.