mlsanityData to 7 Oct 2026

Dance In the Wild: Monocular Human Animation with Neural Dynamic Appearance Synthesis

arXiv 2111.05916 ↗First posted 10 Nov 2021cs.CVcited by 14

Not on the map. Why?

Abstract (arXiv)

Synthesizing dynamic appearances of humans in motion plays a central role in applications such as AR/VR and video editing. While many recent methods have been proposed to tackle this problem, handling loose garments with complex textures and high dynamic motion still remains challenging. In this paper, we propose a video based appearance synthesis method that tackles such challenges and demonstrates high quality results for in-the-wild videos that have not been shown before. Specifically, we adopt a StyleGAN based architecture to the task of person specific video based motion retargeting. We introduce a novel motion signature that is used to modulate the generator weights to capture dynamic appearance changes as well as regularizing the single frame based pose estimates to improve temporal coherency. We evaluate our method on a set of challenging videos and show that our approach achieves state-of-the art performance both qualitatively and quantitatively.

Key facts. Cited by 14 papers in the map (data through 7 Oct 2026). Builds on 35 of its 48 references that are in the map. First posted on arXiv 10 Nov 2021.

Builds on

35 of 48 references.

References · 35

  1. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  2. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
  3. 2016Image-to-Image Translation with Conditional Adversarial Networkscited by 4,510
  4. 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,580
  5. 2016Perceptual Losses for Real-Time Style Transfer and Super-Resolutioncited by 2,678
  6. 2021Alias-Free Generative Adversarial Networkscited by 1,181
  7. 2018OpenPose: Realtime Multi-Person 2D Pose Estimation using Part Affinity Fieldscited by 1,161
  8. 2016FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networkscited by 780
  9. 2020Neural Body: Implicit Neural Representations with Structured Latent Codes for Novel View Synthesis of Dynamic Humanscited by 634
  10. 2018DensePose: Dense Human Pose Estimation In The Wildcited by 573
  11. 2018Everybody Dance Nowcited by 313
  12. 2020State of the Art on Neural Renderingcited by 276
23 more
  1. 2020StyleRig: Rigging StyleGAN for 3D Control over Portrait Imagescited by 267
  2. 2019Deferred Neural Rendering: Image Synthesis using Neural Texturescited by 253
  3. 2017Pose Guided Person Image Generationcited by 251
  4. 2017Deformable GANs for Pose-based Human Image Generationcited by 166
  5. 2019Liquid Warping GAN: A Unified Framework for Human Motion Imitation, Appearance Transfer and Novel View Synthesiscited by 161
  6. 2019Progressive Pose Attention Transfer for Person Image Generationcited by 138
  7. 2021Real-time Deep Dynamic Characterscited by 133
  8. 2017Disentangled Person Image Generationcited by 98
  9. 2018Recycle-GAN: Unsupervised Video Retargetingcited by 89
  10. 2018Dense Pose Transfercited by 83
  11. 2020SMPLpix: Neural Avatars from 3D Human Modelscited by 69
  12. 2019Textured Neural Avatarscited by 65
  13. 2021StylePeople: A Generative Model of Fullbody Human Avatarscited by 62
  14. 2021Neural Re-Rendering of Humans from a Single Imagecited by 59
  15. 2018Unsupervised Person Image Synthesis in Arbitrary Posescited by 55
  16. 2020ANR: Articulated Neural Rendering for Virtual Avatarscited by 55
  17. 2021Style and Pose Control for Image Synthesis of Humans from a Single Monocular Viewcited by 53
  18. 2018Deep Video-Based Performance Cloningcited by 32
  19. 2020High-Fidelity Neural Human Motion Transfer from Monocular Videocited by 31
  20. 2021TryOnGAN: Body-Aware Try-On via Layered Interpolationcited by 31
  21. 2021Dynamic Neural Garmentscited by 27
  22. 2020Neural Human Video Rendering by Learning Dynamic Textures and Rendering-to-Video Translationcited by 13
  23. 2021HumanGPS: Geodesic PreServing Feature for Dense Human Correspondencescited by 12

Built on by

Cited by year

Citing papers per year: 2022: 5; 2023: 8; 2024: 1.

14 papers.

Sort:

By area:

  1. 2022HumanNeRF: Free-viewpoint Rendering of Moving People from Monocular Videocited by 483
  2. 2023LEO: Generative Latent Image Animator for Human Video Synthesiscited by 40
  3. 2023LAC: Latent Action Composition for Skeleton-based Action Segmentationcited by 15
  4. 2022Learning Motion-Dependent Appearance for High-Fidelity Rendering of Dynamic Humans from a Single Cameracited by 12
  5. 2022RANA: Relightable Articulated Neural Avatarscited by 12
  6. 20234DHumanOutfit: a multi-subject 4D dataset of human motion sequences in varying outfits exhibiting large displacementscited by 12
  7. 2023UniHuman: A Unified Model for Editing Human Images in the Wildcited by 12
  8. 2023Generalizable Neural Voxels for Fast Human Radiance Fieldscited by 8
  9. 2023MoRF: Mobile Realistic Fullbody Avatars from a Monocular Videocited by 7
  10. 2023Normal-guided Garment UV Prediction for Human Re-texturingcited by 5
  11. 2022REMOT: A Region-to-Whole Framework for Realistic Human Motion Transfercited by 4
  12. 2022Can We Find Neurons that Cause Unrealistic Images in Deep Generative Networks?cited by 3
2 more
  1. 2024Generalizable Neural Human Renderercited by 3
  2. 2023Bidirectional Temporal Diffusion Model for Temporally Consistent Human Animationcited by 2