mlsanityData to 7 Oct 2026

Cross-identity Video Motion Retargeting with Joint Transformation and Synthesis

arXiv 2210.01559 ↗First posted 2 Oct 2022cs.CVcited by 10

Not on the map. Why?

Abstract (arXiv)

In this paper, we propose a novel dual-branch Transformation-Synthesis network (TS-Net), for video motion retargeting. Given one subject video and one driving video, TS-Net can produce a new plausible video with the subject appearance of the subject video and motion pattern of the driving video. TS-Net consists of a warp-based transformation branch and a warp-free synthesis branch. The novel design of dual branches combines the strengths of deformation-grid-based transformation and warp-free generation for better identity preservation and robustness to occlusion in the synthesized videos. A mask-aware similarity module is further introduced to the transformation branch to reduce computational overhead. Experimental results on face and dance datasets show that TS-Net achieves better performance in video motion retargeting than several state-of-the-art models as well as its single-branch variants. Our code is available at https://github.com/nihaomiao/WACV23_TSNet.

Key facts. Cited by 10 papers in the map (data through 7 Oct 2026). Builds on 51 of its 58 references that are in the map. First posted on arXiv 2 Oct 2022.

Builds on

51 of 58 references.

References · 51

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  4. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  5. 2022Generative Adversarial Networkscited by 5,603
  6. 2016Image-to-Image Translation with Conditional Adversarial Networkscited by 4,510
  7. 2016Perceptual Losses for Real-Time Style Transfer and Super-Resolutioncited by 2,678
  8. 2014Conditional Generative Adversarial Netscited by 2,152
  9. 2015Spatial Transformer Networkscited by 1,423
  10. 2019Semantic Image Synthesis with Spatially-Adaptive Normalizationcited by 1,295
  11. 2016Instance Normalization: The Missing Ingredient for Fast Stylizationcited by 1,187
  12. 2017High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANscited by 1,176
38 more
  1. 2018OpenPose: Realtime Multi-Person 2D Pose Estimation using Part Affinity Fieldscited by 1,161
  2. 2015FlowNet: Learning Optical Flow with Convolutional Networkscited by 1,054
  3. 2016Least Squares Generative Adversarial Networkscited by 1,000
  4. 2020First Order Motion Model for Image Animationcited by 698
  5. 2013Network In Networkcited by 600
  6. 2018DensePose: Dense Human Pose Estimation In The Wildcited by 573
  7. 2020Dynamic Neural Radiance Fields for Monocular 4D Facial Avatar Reconstructioncited by 465
  8. 2020One-Shot Free-View Neural Talking-Head Synthesis for Video Conferencingcited by 438
  9. 2018Video-to-Video Synthesiscited by 388
  10. 2018Everybody Dance Nowcited by 313
  11. 2018An Intriguing Failing of Convolutional Neural Networks and the CoordConv Solutioncited by 302
  12. 2015Deep multi-scale video prediction beyond mean square errorcited by 297
  13. 2019Few-Shot Adversarial Learning of Realistic Neural Talking Head Modelscited by 284
  14. 2018Deep Video Portraitscited by 265
  15. 2021Motion Representations for Articulated Animationcited by 253
  16. 2017Pose Guided Person Image Generationcited by 251
  17. 2018Animating Arbitrary Objects via Deep Motion Transfercited by 206
  18. 2021PIRenderer: Controllable Portrait Image Generation via Semantic Neural Renderingcited by 202
  19. 2019Few-shot Video-to-Video Synthesiscited by 196
  20. 2018X2Face: A network for controlling face generation by using images, audio, and pose codescited by 189
  21. 2019U-GAT-IT: Unsupervised Generative Attentional Networks with Adaptive Layer-Instance Normalization for Image-to-Image Translationcited by 186
  22. 2022Generating Videos with Dynamics-aware Implicit Generative Adversarial Networkscited by 186
  23. 2022Depth-Aware Generative Adversarial Network for Talking Head Video Generationcited by 177
  24. 2017Deformable GANs for Pose-based Human Image Generationcited by 166
  25. 2013A Survey on Multi-view Learningcited by 152
  26. 2018FaceForensics: A Large-scale Video Dataset for Forgery Detection in Human Facescited by 118
  27. 2020Cross-domain Correspondence Learning for Exemplar-based Image Translationcited by 116
  28. 2018Synthesizing Images of Humans in Unseen Posescited by 96
  29. 2018VideoMatch: Matching based Video Object Segmentationcited by 90
  30. 2019MarioNETte: Few-shot Face Reenactment Preserving Identity of Unseen Targetscited by 90
  31. 2018Recycle-GAN: Unsupervised Video Retargetingcited by 89
  32. 2021Neural Re-Rendering of Humans from a Single Imagecited by 59
  33. 2018Unsupervised Person Image Synthesis in Arbitrary Posescited by 55
  34. 2020TransMoMo: Invariance-Driven Unsupervised Video Motion Retargetingcited by 36
  35. 2022Structure-Aware Motion Transfer with Deformable Anchor Modelcited by 34
  36. 2019Unsupervised Person Image Generation with Semantic Parsing Transformationcited by 32
  37. 2019Appearance and Pose-Conditioned Human Image Generation using Deformable GANscited by 24
  38. 2022Coordinate Translator for Learning Deformable Medical Image Registrationcited by 20

50 most cited of 51 All 51 (JSON)

Built on by

Cited by year

Citing papers per year: 2023: 3; 2024: 6; 2025: 1.

10 papers.

  1. 2023Conditional Image-to-Video Generation with Latent Flow Diffusion Modelscited by 180
  2. 2024Magic-Me: Identity-Specific Video Customized Diffusioncited by 58
  3. 2024TI2V-Zero: Zero-Shot Image Conditioning for Text-to-Video Diffusion Modelscited by 25
  4. 2024Dance Any Beat: Blending Beats with Visuals in Dance Video Generationcited by 14
  5. 2023From Pixels to Portraits: A Comprehensive Survey of Talking Head Generation Techniques and Applicationscited by 9
  6. 2024Reenact Anything: Semantic Video Motion Transfer Using Motion-Textual Inversioncited by 6
  7. 2025Advancing Talking Head Generation: A Comprehensive Survey of Multi-Modal Methodologies, Datasets, Evaluation Metrics, and Loss Functionscited by 6
  8. 20233D-Aware Talking-Head Video Motion Transfercited by 4
  9. 2024A Comprehensive Taxonomy and Analysis of Talking Head Synthesis: Techniques for Portrait Generation, Driving Mechanisms, and Editingcited by 4
  10. 2024One-Shot Pose-Driving Face Animation Platformcited by 2