# Do As I Do: Pose Guided Human Motion Copy

arXiv: https://arxiv.org/abs/2406.16601 · HTML: https://mlsanity.com/paper/2406.16601/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 42 of its 69 references that are in the map.
- First posted on arXiv 24 Jun 2024.

## Builds on (most cited first, 30 of 42)

- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [Improved Techniques for Training GANs](https://mlsanity.com/paper/1606.03498/) (2016)
- [Improved Training of Wasserstein GANs](https://mlsanity.com/paper/1704.00028/) (2017)
- [Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields](https://mlsanity.com/paper/1611.08050/) (2016)
- [Pixel Recurrent Neural Networks](https://mlsanity.com/paper/1601.06759/) (2016)
- [Free-Form Image Inpainting with Gated Convolution](https://mlsanity.com/paper/1806.03589/) (2018)
- [StackGAN: Text to Photo-realistic Image Synthesis with Stacked Generative Adversarial Networks](https://mlsanity.com/paper/1612.03242/) (2016)
- [MoCoGAN: Decomposing Motion and Content for Video Generation](https://mlsanity.com/paper/1707.04993/) (2017)
- [Towards Principled Methods for Training Generative Adversarial Networks](https://mlsanity.com/paper/1701.04862/) (2017)
- [Video-to-Video Synthesis](https://mlsanity.com/paper/1808.06601/) (2018)
- [Everybody Dance Now](https://mlsanity.com/paper/1808.07371/) (2018)
- [Pose Guided Person Image Generation](https://mlsanity.com/paper/1705.09368/) (2017)
- [Few-shot Video-to-Video Synthesis](https://mlsanity.com/paper/1910.12713/) (2019)
- [Episodic Memory in Lifelong Language Learning](https://mlsanity.com/paper/1906.01076/) (2019)
- [Low Dose CT Image Denoising Using a Generative Adversarial Network with Wasserstein Distance and Perceptual Loss](https://mlsanity.com/paper/1708.00961/) (2017)
- [Liquid Warping GAN: A Unified Framework for Human Motion Imitation, Appearance Transfer and Novel View Synthesis](https://mlsanity.com/paper/1909.12224/) (2019)
- [A Variational U-Net for Conditional Appearance and Shape Generation](https://mlsanity.com/paper/1804.04694/) (2018)
- [Synthesizing Images of Humans in Unseen Poses](https://mlsanity.com/paper/1804.07739/) (2018)
- [Recycle-GAN: Unsupervised Video Retargeting](https://mlsanity.com/paper/1808.05174/) (2018)
- [Textured Neural Avatars](https://mlsanity.com/paper/1905.08776/) (2019)
- [PISE: Person Image Synthesis and Editing with Decoupled GAN](https://mlsanity.com/paper/2103.04023/) (2021)
- [Soft-Gated Warping-GAN for Pose-Guided Person Image Synthesis](https://mlsanity.com/paper/1810.11610/) (2018)
- [Unsupervised Person Image Synthesis in Arbitrary Poses](https://mlsanity.com/paper/1809.10280/) (2018)
- [Deep Dual Consecutive Network for Human Pose Estimation](https://mlsanity.com/paper/2103.07254/) (2021)
- [TransMoMo: Invariance-Driven Unsupervised Video Motion Retargeting](https://mlsanity.com/paper/2003.14401/) (2020)
- [Investigating Pose Representations and Motion Contexts Modeling for 3D Motion Prediction](https://mlsanity.com/paper/2112.15012/) (2021)

## Built on by (most cited first, 8 of 8)

- [Exploring Efficient Open-Vocabulary Segmentation in the Remote Sensing](https://mlsanity.com/paper/2509.12040/) (2025)
- [Human Motion Video Generation: A Survey](https://mlsanity.com/paper/2509.03883/) (2025)
- [Joint-Motion Mutual Learning for Pose Estimation in Videos](https://mlsanity.com/paper/2408.02285/) (2024)
- [MCMoE: Completing Missing Modalities with Mixture of Experts for Incomplete Multimodal Action Quality Assessment](https://mlsanity.com/paper/2511.17397/) (2025)
- [SpatioTemporal Learning for Human Pose Estimation in Sparsely-Labeled Videos](https://mlsanity.com/paper/2501.15073/) (2025)
- [Cross-modal Proxy Evolving for OOD Detection with Vision-Language Models](https://mlsanity.com/paper/2601.08476/) (2026)
- [A Pioneering Neural Network Method for Efficient and Robust Fluid Simulation](https://mlsanity.com/paper/2412.10748/) (2024)
- [Multi-Grained Feature Pruning for Video-Based Human Pose Estimation](https://mlsanity.com/paper/2503.05365/) (2025)

## Abstract (arXiv)

Human motion copy is an intriguing yet challenging task in artificial intelligence and computer vision, which strives to generate a fake video of a target person performing the motion of a source person. The problem is inherently challenging due to the subtle human-body texture details to be generated and the temporal consistency to be considered. Existing approaches typically adopt a conventional GAN with an L1 or L2 loss to produce the target fake video, which intrinsically necessitates a large number of training samples that are challenging to acquire. Meanwhile, current methods still have difficulties in attaining realistic image details and temporal consistency, which unfortunately can be easily perceived by human observers. Motivated by this, we try to tackle the issues from three aspects: (1) We constrain pose-to-appearance generation with a perceptual loss and a theoretically motivated Gromov-Wasserstein loss to bridge the gap between pose and appearance. (2) We present an episodic memory module in the pose-to-appearance generation to propel continuous learning that helps the model learn from its past poor generations. We also utilize geometrical cues of the face to optimize facial details and refine each key body part with a dedicated local GAN. (3) We advocate generating the foreground in a sequence-to-sequence manner rather than a single-frame manner, explicitly enforcing temporal inconsistency. Empirical results on five datasets, iPER, ComplexMotion, SoloDance, Fish, and Mouse datasets, demonstrate that our method is capable of generating realistic target videos while precisely copying motion from a source video. Our method significantly outperforms state-of-the-art approaches and gains 7.2% and 12.4% improvements in PSNR and FID respectively.