# Structure-Aware Motion Transfer with Deformable Anchor Model

arXiv: https://arxiv.org/abs/2204.05018 · HTML: https://mlsanity.com/paper/2204.05018/

## Key facts

- Cited by 34 papers in the map (data through 14 Oct 2026).
- Builds on 39 of its 43 references that are in the map.
- First posted on arXiv 11 Apr 2022.

## Builds on (most cited first, 30 of 39)

- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields](https://mlsanity.com/paper/1611.08050/) (2016)
- [Stacked Hourglass Networks for Human Pose Estimation](https://mlsanity.com/paper/1603.06937/) (2016)
- [VoxCeleb: a large-scale speaker identification dataset](https://mlsanity.com/paper/1706.08612/) (2017)
- [First Order Motion Model for Image Animation](https://mlsanity.com/paper/2003.00196/) (2020)
- [One-Shot Free-View Neural Talking-Head Synthesis for Video Conferencing](https://mlsanity.com/paper/2011.15126/) (2020)
- [Video-to-Video Synthesis](https://mlsanity.com/paper/1808.06601/) (2018)
- [Everybody Dance Now](https://mlsanity.com/paper/1808.07371/) (2018)
- [Deep Video Portraits](https://mlsanity.com/paper/1805.11714/) (2018)
- [Motion Representations for Articulated Animation](https://mlsanity.com/paper/2104.11280/) (2021)
- [Pose Guided Person Image Generation](https://mlsanity.com/paper/1705.09368/) (2017)
- [Animating Arbitrary Objects via Deep Motion Transfer](https://mlsanity.com/paper/1812.08861/) (2018)
- [X2Face: A network for controlling face generation by using images, audio, and pose codes](https://mlsanity.com/paper/1807.10550/) (2018)
- [Deformable GANs for Pose-based Human Image Generation](https://mlsanity.com/paper/1801.00055/) (2017)
- [GANimation: Anatomically-aware Facial Animation from a Single Image](https://mlsanity.com/paper/1807.09251/) (2018)
- [Liquid Warping GAN: A Unified Framework for Human Motion Imitation, Appearance Transfer and Novel View Synthesis](https://mlsanity.com/paper/1909.12224/) (2019)
- [Progressive Pose Attention Transfer for Person Image Generation](https://mlsanity.com/paper/1904.03349/) (2019)
- [DwNet: Dense warp-based network for pose-guided human video generation](https://mlsanity.com/paper/1910.09139/) (2019)
- [Unsupervised Learning of Object Landmarks through Conditional Image Generation](https://mlsanity.com/paper/1806.07823/) (2018)
- [Neural Head Reenactment with Latent Pose Descriptors](https://mlsanity.com/paper/2004.12000/) (2020)
- [Dense Intrinsic Appearance Flow for Human Pose Transfer](https://mlsanity.com/paper/1903.11326/) (2019)
- [Deep Image Spatial Transformation for Person Image Generation](https://mlsanity.com/paper/2003.00696/) (2020)
- [Disentangled Person Image Generation](https://mlsanity.com/paper/1712.02621/) (2017)
- [Synthesizing Images of Humans in Unseen Poses](https://mlsanity.com/paper/1804.07739/) (2018)
- [Dense Pose Transfer](https://mlsanity.com/paper/1809.01995/) (2018)
- [Unsupervised Discovery of Object Landmarks as Structural Representations](https://mlsanity.com/paper/1804.04412/) (2018)
- [PISE: Person Image Synthesis and Editing with Decoupled GAN](https://mlsanity.com/paper/2103.04023/) (2021)
- [Neural Re-Rendering of Humans from a Single Image](https://mlsanity.com/paper/2101.04104/) (2021)
- [Pose-Guided Human Animation from a Single Image in the Wild](https://mlsanity.com/paper/2012.03796/) (2020)
- [TransMoMo: Invariance-Driven Unsupervised Video Motion Retargeting](https://mlsanity.com/paper/2003.14401/) (2020)

## Built on by (most cited first, 30 of 34)

- [Space-Time Diffusion Features for Zero-Shot Text-Driven Motion Transfer](https://mlsanity.com/paper/2311.17009/) (2023)
- [MetaPortrait: Identity-Preserving Talking Head Generation with Fast Personalized Adaptation](https://mlsanity.com/paper/2212.08062/) (2022)
- [Human-Art: A Versatile Human-Centric Dataset Bridging Natural and Artificial Scenes](https://mlsanity.com/paper/2303.02760/) (2023)
- [Face Adapter for Pre-Trained Diffusion Models with Fine-Grained ID and Attribute Control](https://mlsanity.com/paper/2405.12970/) (2024)
- [Implicit Identity Representation Conditioned Memory Compensation Network for Talking Head video Generation](https://mlsanity.com/paper/2307.09906/) (2023)
- [Magic-Me: Identity-Specific Video Customized Diffusion](https://mlsanity.com/paper/2402.09368/) (2024)
- [VOODOO 3D: Volumetric Portrait Disentanglement for One-Shot 3D Head Reenactment](https://mlsanity.com/paper/2312.04651/) (2023)
- [Text2Human: Text-Driven Controllable Human Image Generation](https://mlsanity.com/paper/2205.15996/) (2022)
- [VOODOO XP: Expressive One-Shot Head Reenactment for VR Telepresence](https://mlsanity.com/paper/2405.16204/) (2024)
- [Unsupervised Volumetric Animation](https://mlsanity.com/paper/2301.11326/) (2023)
- [Human MotionFormer: Transferring Human Motions with Vision Transformers](https://mlsanity.com/paper/2302.11306/) (2023)
- [A Generalist FaceX via Learning Unified Facial Representation](https://mlsanity.com/paper/2401.00551/) (2023)
- [Cross-identity Video Motion Retargeting with Joint Transformation and Synthesis](https://mlsanity.com/paper/2210.01559/) (2022)
- [Multimodal-driven Talking Face Generation via a Unified Diffusion-based Generator](https://mlsanity.com/paper/2305.02594/) (2023)
- [From Pixels to Portraits: A Comprehensive Survey of Talking Head Generation Techniques and Applications](https://mlsanity.com/paper/2308.16041/) (2023)
- [Motion Transformer for Unsupervised Image Animation](https://mlsanity.com/paper/2209.14024/) (2022)
- [Pose-Guided Fine-Grained Sign Language Video Generation](https://mlsanity.com/paper/2409.16709/) (2024)
- [Learning Motion Refinement for Unsupervised Face Animation](https://mlsanity.com/paper/2310.13912/) (2023)
- [Enhancing Sketch Animation: Text-to-Video Diffusion Models with Temporal Consistency and Rigidity Constraints](https://mlsanity.com/paper/2411.19381/) (2024)
- [EFHQ: Multi-purpose ExtremePose-Face-HQ dataset](https://mlsanity.com/paper/2312.17205/) (2023)
- [Motion and Appearance Adaptation for Cross-Domain Motion Transfer](https://mlsanity.com/paper/2209.14529/) (2022)
- [Real-time Animation Generation and Control on Rigged Models via Large Language Models](https://mlsanity.com/paper/2310.17838/) (2023)
- [Generative Human Video Compression with Multi-granularity Temporal Trajectory Factorization](https://mlsanity.com/paper/2410.10171/) (2024)
- [Free-viewpoint Human Animation with Pose-correlated Reference Selection](https://mlsanity.com/paper/2412.17290/) (2024)
- [3D-Aware Talking-Head Video Motion Transfer](https://mlsanity.com/paper/2311.02549/) (2023)
- [AnaMoDiff: 2D Analogical Motion Diffusion via Disentangled Denoising](https://mlsanity.com/paper/2402.03549/) (2024)
- [An Identity-Preserved Framework for Human Motion Transfer](https://mlsanity.com/paper/2204.06862/) (2022)
- [MIMAFace: Face Animation via Motion-Identity Modulated Appearance Feature Learning](https://mlsanity.com/paper/2409.15179/) (2024)
- [ImPoster: Text and Frequency Guidance for Subject Driven Action Personalization using Diffusion Models](https://mlsanity.com/paper/2409.15650/) (2024)
- [One-shot Human Motion Transfer via Occlusion-Robust Flow Prediction and Neural Texturing](https://mlsanity.com/paper/2412.06174/) (2024)

## Abstract (arXiv)

Given a source image and a driving video depicting the same object type, the motion transfer task aims to generate a video by learning the motion from the driving video while preserving the appearance from the source image. In this paper, we propose a novel structure-aware motion modeling approach, the deformable anchor model (DAM), which can automatically discover the motion structure of arbitrary objects without leveraging their prior structure information. Specifically, inspired by the known deformable part model (DPM), our DAM introduces two types of anchors or keypoints: i) a number of motion anchors that capture both appearance and motion information from the source image and driving video; ii) a latent root anchor, which is linked to the motion anchors to facilitate better learning of the representations of the object structure information. Moreover, DAM can be further extended to a hierarchical version through the introduction of additional latent anchors to model more complicated structures. By regularizing motion anchors with latent anchor(s), DAM enforces the correspondences between them to ensure the structural information is well captured and preserved. Moreover, DAM can be learned effectively in an unsupervised manner. We validate our proposed DAM for motion transfer on different benchmark datasets. Extensive experiments clearly demonstrate that DAM achieves superior performance relative to existing state-of-the-art methods.