mlsanityData to 14 Oct 2026

Do as we do: Multiple Person Video-To-Video Transfer

arXiv 2104.04721 ↗First posted 10 Apr 2021cs.CVcited by 1

Not on the map. Why?

Abstract (arXiv)

Our goal is to transfer the motion of real people from a source video to a target video with realistic results. While recent advances significantly improved image-to-image translations, only few works account for body motions and temporal consistency. However, those focus only on video re-targeting for a single actor/ for single actors. In this work, we propose a marker-less approach for multiple-person video-to-video transfer using pose as an intermediate representation. Given a source video with multiple persons dancing or working out, our method transfers the body motion of all actors to a new set of actors in a different video. Differently from recent "do as I do" methods, we focus specifically on transferring multiple person at the same time and tackle the related identity switch problem. Our method is able to convincingly transfer body motion to the target video, while preserving specific features of the target video, such as feet touching the floor and relative position of the actors. The evaluation is performed with visual quality and appearance metrics using publicly available videos with the permission of their owners.

Key facts. Cited by 1 paper in the map (data through 14 Oct 2026). Builds on 27 of its 37 references that are in the map. First posted on arXiv 10 Apr 2021.

Builds on

27 of 37 references.

References · 27

  1. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,177
  2. 2016Image-to-Image Translation with Conditional Adversarial Networkscited by 4,527
  3. 2017High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANscited by 1,179
  4. 2018OpenPose: Realtime Multi-Person 2D Pose Estimation using Part Affinity Fieldscited by 1,162
  5. 2016Realtime Multi-Person 2D Pose Estimation using Part Affinity Fieldscited by 1,064
  6. 2016StackGAN: Text to Photo-realistic Image Synthesis with Stacked Generative Adversarial Networkscited by 577
  7. 2016Conditional Image Synthesis With Auxiliary Classifier GANscited by 565
  8. 2016Convolutional Pose Machinescited by 325
  9. 2018Everybody Dance Nowcited by 315
  10. 2017Hand Keypoint Detection in Single Images using Multiview Bootstrappingcited by 231
  11. 2019Few-shot Video-to-Video Synthesiscited by 197
  12. 2020Controllable Person Image Synthesis with Attribute-Decomposed GANcited by 150
15 more
  1. 2015Attribute2Image: Conditional Image Generation from Visual Attributescited by 93
  2. 2018Learning to Detect and Track Visible and Occluded Body Joints in a Virtual Worldcited by 82
  3. 2017Be Your Own Prada: Fashion Synthesis with Structural Coherencecited by 49
  4. 2020ARID: A New Dataset for Recognizing Action in the Darkcited by 43
  5. 2017A Generative Model of People in Clothingcited by 37
  6. 2018Exploring Spatial Context for 3D Semantic Segmentation of Point Cloudscited by 33
  7. 2020Learning to dance: A graph convolutional adversarial network to generate realistic dance motions from audiocited by 33
  8. 2019Dance Dance Generation: Motion Transfer for Internet Videoscited by 28
  9. 2020Self-supervised Keypoint Correspondences for Multi-Person Pose Estimation and Tracking in Videoscited by 22
  10. 2020Human Motion Transfer from Poses in the Wildcited by 17
  11. 2019Human Pose Estimation for Real-World Crowded Scenarioscited by 9
  12. 2019Video synthesis of human upper body with realistic facecited by 4
  13. 2020Pose-Guided High-Resolution Appearance Transfer via Progressive Trainingcited by 4
  14. 2020Do As I Do: Transferring Human Motion and Appearance between Monocular Videos with Spatial and Temporal Constraintscited by 3
  15. 2020From Real to Synthetic and Back: Synthesizing Training Data for Multi-Person Scene Understandingcited by 3

Built on by

Cited by year

Citing papers per year: 2022: 1.

1 paper.

  1. 2022CTrGAN: Cycle Transformers GAN for Gait Transfercited by 1