# Towards Robust 3D Pose Transfer with Adversarial Learning

arXiv: https://arxiv.org/abs/2404.02242 · HTML: https://mlsanity.com/paper/2404.02242/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 45 of its 57 references that are in the map.
- First posted on arXiv 2 Apr 2024.

## Builds on (most cited first, 30 of 45)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [Explaining and Harnessing Adversarial Examples](https://mlsanity.com/paper/1412.6572/) (2014)
- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space](https://mlsanity.com/paper/1706.02413/) (2017)
- [Towards Evaluating the Robustness of Neural Networks](https://mlsanity.com/paper/1608.04644/) (2016)
- [Arbitrary Style Transfer in Real-time with Adaptive Instance Normalization](https://mlsanity.com/paper/1703.06868/) (2017)
- [Expressive Body Capture: 3D Hands, Face, and Body from a Single Image](https://mlsanity.com/paper/1904.05866/) (2019)
- [Point Transformer](https://mlsanity.com/paper/2011.00931/) (2020)
- [Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attacks](https://mlsanity.com/paper/2003.01690/) (2020)
- [Universal adversarial perturbations](https://mlsanity.com/paper/1610.08401/) (2016)
- [Keep it SMPL: Automatic Estimation of 3D Human Pose and Shape from a Single Image](https://mlsanity.com/paper/1607.08128/) (2016)
- [One pixel attack for fooling deep neural networks](https://mlsanity.com/paper/1710.08864/) (2017)
- [Masked Autoencoders for Point Cloud Self-supervised Learning](https://mlsanity.com/paper/2203.06604/) (2022)
- [On Evaluating Adversarial Robustness](https://mlsanity.com/paper/1902.06705/) (2019)
- [PoinTr: Diverse Point Cloud Completion with Geometry-Aware Transformers](https://mlsanity.com/paper/2108.08839/) (2021)
- [Point-M2AE: Multi-scale Masked Autoencoders for Hierarchical Point Cloud Pre-training](https://mlsanity.com/paper/2205.14401/) (2022)
- [3D-CODED : 3D Correspondences by Deep Deformation](https://mlsanity.com/paper/1806.05228/) (2018)
- [Generating 3D Adversarial Point Clouds](https://mlsanity.com/paper/1809.07016/) (2018)
- [Skeleton-Aware Networks for Deep Motion Retargeting](https://mlsanity.com/paper/2005.05732/) (2020)
- [PointCloud Saliency Maps](https://mlsanity.com/paper/1812.01687/) (2018)
- [Extending Adversarial Attacks and Defenses to Deep 3D Point Cloud Classifiers](https://mlsanity.com/paper/1901.03006/) (2019)
- [The Power of Points for Modeling Humans in Clothing](https://mlsanity.com/paper/2109.01137/) (2021)
- [MetaAvatar: Learning Animatable Clothed Human Models from Few Depth Images](https://mlsanity.com/paper/2106.11944/) (2021)
- [Unsupervised Shape and Pose Disentanglement for 3D Meshes](https://mlsanity.com/paper/2007.11341/) (2020)
- [LIMP: Learning Latent Shape Representations with Metric Preservation Priors](https://mlsanity.com/paper/2003.12283/) (2020)
- [RDA: Robust Domain Adaptation via Fourier Adversarial Attacking](https://mlsanity.com/paper/2106.02874/) (2021)
- [IF-Defense: 3D Adversarial Point Cloud Defense via Implicit Function based Restoration](https://mlsanity.com/paper/2010.05272/) (2020)
- [Minimal Adversarial Examples for Deep Learning on 3D Point Clouds](https://mlsanity.com/paper/2008.12066/) (2020)

## Built on by (most cited first, 4 of 4)

- [Auto-Connect: Connectivity-Preserving RigFormer with Direct Preference Optimization](https://mlsanity.com/paper/2506.11430/) (2025)
- [Unveiling Hidden Vulnerabilities in Digital Human Generation via Adversarial Attacks](https://mlsanity.com/paper/2504.17457/) (2025)
- [MimiCAT: Mimic with Correspondence-Aware Cascade-Transformer for Category-Free 3D Pose Transfer](https://mlsanity.com/paper/2511.18370/) (2025)
- [LatentStealth: Unnoticeable and Efficient Adversarial Attacks on Expressive Human Pose and Shape Estimation](https://mlsanity.com/paper/2505.12009/) (2025)

## Abstract (arXiv)

3D pose transfer that aims to transfer the desired pose to a target mesh is one of the most challenging 3D generation tasks. Previous attempts rely on well-defined parametric human models or skeletal joints as driving pose sources. However, to obtain those clean pose sources, cumbersome but necessary pre-processing pipelines are inevitable, hindering implementations of the real-time applications. This work is driven by the intuition that the robustness of the model can be enhanced by introducing adversarial samples into the training, leading to a more invulnerable model to the noisy inputs, which even can be further extended to directly handling the real-world data like raw point clouds/scans without intermediate processing. Furthermore, we propose a novel 3D pose Masked Autoencoder (3D-PoseMAE), a customized MAE that effectively learns 3D extrinsic presentations (i.e., pose). 3D-PoseMAE facilitates learning from the aspect of extrinsic attributes by simultaneously generating adversarial samples that perturb the model and learning the arbitrary raw noisy poses via a multi-scale masking strategy. Both qualitative and quantitative studies show that the transferred meshes given by our network result in much better quality. Besides, we demonstrate the strong generalizability of our method on various poses, different domains, and even raw scans. Experimental results also show meaningful insights that the intermediate adversarial samples generated in the training can successfully attack the existing pose transfer models.