# A Shape-Aware Retargeting Approach to Transfer Human Motion and Appearance in Monocular Videos arXiv: https://arxiv.org/abs/2103.15596 ยท HTML: https://mlsanity.com/paper/2103.15596/ ## Key facts - Cited by 8 papers in the map (data through 14 Oct 2026). - Builds on 43 of its 57 references that are in the map. - First posted on arXiv 29 Mar 2021. ## Builds on (most cited first, 30 of 43) - [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014) - [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018) - [AMASS: Archive of Motion Capture as Surface Shapes](https://mlsanity.com/paper/1904.03278/) (2019) - [Towards Accurate Generative Models of Video: A New Metric & Challenges](https://mlsanity.com/paper/1812.01717/) (2018) - [Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields](https://mlsanity.com/paper/1611.08050/) (2016) - [End-to-end Recovery of Human Shape and Pose](https://mlsanity.com/paper/1712.06584/) (2017) - [CNN-generated images are surprisingly easy to spot... for now](https://mlsanity.com/paper/1912.11035/) (2019) - [Keep it SMPL: Automatic Estimation of 3D Human Pose and Shape from a Single Image](https://mlsanity.com/paper/1607.08128/) (2016) - [Learning to Reconstruct 3D Human Pose and Shape via Model-fitting in the Loop](https://mlsanity.com/paper/1909.12828/) (2019) - [Free-Form Image Inpainting with Gated Convolution](https://mlsanity.com/paper/1806.03589/) (2018) - [Generative Image Inpainting with Contextual Attention](https://mlsanity.com/paper/1801.07892/) (2018) - [Monocular 3D Human Pose Estimation In The Wild Using Improved CNN Supervision](https://mlsanity.com/paper/1611.09813/) (2016) - [Video-to-Video Synthesis](https://mlsanity.com/paper/1808.06601/) (2018) - [Convolutional Pose Machines](https://mlsanity.com/paper/1602.00134/) (2016) - [Everybody Dance Now](https://mlsanity.com/paper/1808.07371/) (2018) - [Resolving 3D Human Pose Ambiguities with 3D Scene Constraints](https://mlsanity.com/paper/1908.06963/) (2019) - [State of the Art on Neural Rendering](https://mlsanity.com/paper/2004.03805/) (2020) - [Pose Guided Person Image Generation](https://mlsanity.com/paper/1705.09368/) (2017) - [Video Based Reconstruction of 3D People Models](https://mlsanity.com/paper/1803.04758/) (2018) - [Hand Keypoint Detection in Single Images using Multiview Bootstrapping](https://mlsanity.com/paper/1704.07809/) (2017) - [Unite the People: Closing the Loop Between 3D and 2D Human Representations](https://mlsanity.com/paper/1701.02468/) (2017) - [Liquid Warping GAN: A Unified Framework for Human Motion Imitation, Appearance Transfer and Novel View Synthesis](https://mlsanity.com/paper/1909.12224/) (2019) - [Seeing What a GAN Cannot Generate](https://mlsanity.com/paper/1910.11626/) (2019) - [Instance-level Human Parsing via Part Grouping Network](https://mlsanity.com/paper/1808.00157/) (2018) - [A Variational U-Net for Conditional Appearance and Shape Generation](https://mlsanity.com/paper/1804.04694/) (2018) - [Deep Flow-Guided Video Inpainting](https://mlsanity.com/paper/1905.02884/) (2019) - [Disentangled Person Image Generation](https://mlsanity.com/paper/1712.02621/) (2017) - [Synthesizing Images of Humans in Unseen Poses](https://mlsanity.com/paper/1804.07739/) (2018) - [Neural Kinematic Networks for Unsupervised Motion Retargetting](https://mlsanity.com/paper/1804.05653/) (2018) - [Dense Pose Transfer](https://mlsanity.com/paper/1809.01995/) (2018) ## Built on by (most cited first, 8 of 8) - [Skinned Motion Retargeting with Residual Perception of Motion Semantics & Geometry](https://mlsanity.com/paper/2303.08658/) (2023) - [HUMOS: Human Motion Model Conditioned on Body Shape](https://mlsanity.com/paper/2409.03944/) (2024) - [Deep learning for 3D human pose estimation and mesh recovery: A survey](https://mlsanity.com/paper/2402.18844/) (2024) - [Shape My Moves: Text-Driven Shape-Aware Synthesis of Human Motions](https://mlsanity.com/paper/2504.03639/) (2025) - [H-MoRe: Learning Human-centric Motion Representation for Action Analysis](https://mlsanity.com/paper/2504.10676/) (2025) - [Creating and Reenacting Controllable 3D Humans with Differentiable Rendering](https://mlsanity.com/paper/2110.11746/) (2021) - [From Easy to Hard: Learning Curricular Shape-aware Features for Robust Panoptic Scene Graph Generation](https://mlsanity.com/paper/2407.09191/) (2024) - [An Identity-Preserved Framework for Human Motion Transfer](https://mlsanity.com/paper/2204.06862/) (2022) ## Abstract (arXiv) Transferring human motion and appearance between videos of human actors remains one of the key challenges in Computer Vision. Despite the advances from recent image-to-image translation approaches, there are several transferring contexts where most end-to-end learning-based retargeting methods still perform poorly. Transferring human appearance from one actor to another is only ensured when a strict setup has been complied, which is generally built considering their training regime's specificities. In this work, we propose a shape-aware approach based on a hybrid image-based rendering technique that exhibits competitive visual retargeting quality compared to state-of-the-art neural rendering approaches. The formulation leverages the user body shape into the retargeting while considering physical constraints of the motion in 3D and the 2D image domain. We also present a new video retargeting benchmark dataset composed of different videos with annotated human motions to evaluate the task of synthesizing people's videos, which can be used as a common base to improve tracking the progress in the field. The dataset and its evaluation protocols are designed to evaluate retargeting methods in more general and challenging conditions. Our method is validated in several experiments, comprising publicly available videos of actors with different shapes, motion types, and camera setups. The dataset and retargeting code are publicly available to the community at: https://www.verlab.dcc.ufmg.br/retargeting-motion.