mlsanityData to 7 Oct 2026

MagicPose: Realistic Human Poses and Facial Expressions Retargeting with Identity-aware Diffusion

arXiv 2311.12052 ↗First posted 18 Nov 2023cs.CVcited by 115

On the map: Diffusion models → Image generation, text-to-image and editing

Abstract (arXiv)

In this work, we propose MagicPose, a diffusion-based model for 2D human pose and facial expression retargeting. Specifically, given a reference image, we aim to generate a person's new images by controlling the poses and facial expressions while keeping the identity unchanged. To this end, we propose a two-stage training strategy to disentangle human motions and appearance (e.g., facial expressions, skin tone and dressing), consisting of (1) the pre-training of an appearance-control block and (2) learning appearance-disentangled pose control. Our novel design enables robust appearance control over generated human images, including body, facial attributes, and even background. By leveraging the prior knowledge of image diffusion models, MagicPose generalizes well to unseen human identities and complex poses without the need for additional fine-tuning. Moreover, the proposed model is easy to use and can be considered as a plug-in module/extension to Stable Diffusion. The code is available at: https://github.com/Boese0601/MagicDance

Key facts. Cited by 115 papers in the map (data through 7 Oct 2026). Placed in Image generation, text-to-image and editing, one of 14 directions in Diffusion models. Builds on 55 of its 72 references that are in the map. First posted on arXiv 18 Nov 2023.

Builds on

55 of 72 references.

Closest ancestors · 5

Same direction, newest first

  1. 2023Adding Conditional Control to Text-to-Image Diffusion Models10 Feb
  2. 2022eDiff-I: Text-to-Image Diffusion Models with an Ensemble of Expert Denoisers2 Nov
  3. 2022Hierarchical Text-Conditional Image Generation with CLIP Latents13 Apr
  4. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models20 Dec
  5. 2021Palette: Image-to-Image Diffusion Models10 Nov

Other references · 50

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  3. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  4. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  5. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  6. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  7. 2020Denoising Diffusion Implicit Modelscited by 9,655
  8. 2020Score-Based Generative Modeling through Stochastic Differential Equationscited by 8,964
  9. 2022Classifier-Free Diffusion Guidancecited by 5,717
  10. 2017Neural Discrete Representation Learningcited by 4,650
  11. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,611
  12. 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,006
38 more
  1. 2014Conditional Generative Adversarial Netscited by 2,152
  2. 2022Imagen Video: High Definition Video Generation with Diffusion Modelscited by 1,765
  3. 2022Make-A-Video: Text-to-Video Generation without Text-Video Datacited by 1,725
  4. 2021LAION-400M: Open Dataset of CLIP-Filtered 400 Million Image-Text Pairscited by 1,544
  5. 2023Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Modelscited by 1,465
  6. 2023AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuningcited by 1,418
  7. 2018OpenPose: Realtime Multi-Person 2D Pose Estimation using Part Affinity Fieldscited by 1,161
  8. 2016Realtime Multi-Person 2D Pose Estimation using Part Affinity Fieldscited by 1,061
  9. 2022Tune-A-Video: One-Shot Tuning of Image Diffusion Models for Text-to-Video Generationcited by 958
  10. 2023MVDream: Multi-view Diffusion for 3D Generationcited by 928
  11. 2023MasaCtrl: Tuning-Free Mutual Self-Attention Control for Consistent Image Synthesis and Editingcited by 722
  12. 2020First Order Motion Model for Image Animationcited by 698
  13. 2017MoCoGAN: Decomposing Motion and Content for Video Generationcited by 523
  14. 2021Label-Efficient Semantic Segmentation with Diffusion Modelscited by 495
  15. 2022AdaFace: Quality Adaptive Margin for Face Recognitioncited by 390
  16. 2018Everybody Dance Nowcited by 313
  17. 2023Common Diffusion Noise Schedules and Sample Steps are Flawedcited by 308
  18. 2021SegDiff: Image Segmentation with Diffusion Probabilistic Modelscited by 279
  19. 2018Deep Video Portraitscited by 265
  20. 2021Motion Representations for Articulated Animationcited by 253
  21. 2023DreamBooth3D: Subject-Driven Text-to-3D Generationcited by 235
  22. 2017Hand Keypoint Detection in Single Images using Multiview Bootstrappingcited by 231
  23. 2022Thin-Plate Spline Motion Model for Image Animationcited by 209
  24. 2021Diffusion Models for Implicit Image Segmentation Ensemblescited by 198
  25. 2022Depth-Aware Generative Adversarial Network for Talking Head Video Generationcited by 177
  26. 2023DreamPose: Fashion Image-to-Video Synthesis via Stable Diffusioncited by 167
  27. 2017Deformable GANs for Pose-based Human Image Generationcited by 166
  28. 2019Liquid Warping GAN: A Unified Framework for Human Motion Imitation, Appearance Transfer and Novel View Synthesiscited by 161
  29. 2016Semantic Image Inpainting with Deep Generative Modelscited by 156
  30. 2022StyleGAN-Human: A Data-Centric Odyssey of Human Generationcited by 143
  31. 2019DeepFashion2: A Versatile Benchmark for Detection, Pose Estimation, Segmentation and Re-Identification of Clothing Imagescited by 132
  32. 2023Consistent123: One Image to Highly Consistent 3D Asset Using Case-Aware Diffusion Priorscited by 64
  33. 2020Cascade EF-GAN: Progressive Facial Expression Editing with Local Focusescited by 43
  34. 2023A Theoretical Justification for Image Inpainting using Denoising Diffusion Probabilistic Modelscited by 37
  35. 2019Appearance and Pose-Conditioned Human Image Generation using Deformable GANscited by 24
  36. 2018Geometry-Contrastive GAN for Facial Expression Transfercited by 16
  37. 2019GAC-GAN: A General Method for Appearance-Controllable Human Video Motion Transfercited by 14
  38. 2020Human Motion Transfer with 3D Constraints and Detail Enhancementcited by 14

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 32; 2025: 58; 2026: 25.

115 papers.

Sort:

By area:

  1. 2024LivePortrait: Efficient Portrait Animation with Stitching and Retargeting Controlcited by 202
  2. 2024MimicMotion: High-Quality Human Motion Video Generation with Confidence-aware Pose Guidancecited by 196
  3. 2024Follow-Your-Emoji: Fine-Controllable and Expressive Freestyle Portrait Animationcited by 172
  4. 2024ControlNeXt: Powerful and Efficient Control for Image and Video Generationcited by 158
  5. 2025Controllable Video Generation: A Surveycited by 93
  6. 2024X-Portrait: Expressive Portrait Animation with Hierarchical Motion Attentioncited by 88
  7. 2024Animate-X: Universal Character Image Animation with Enhanced Motion Representationcited by 86
  8. 2024OpenHumanVid: A Large-Scale High-Quality Dataset for Enhancing Human-Centric Video Generationcited by 77
  9. 2025Wan-Animate: Unified Character Animation and Replacement with Holistic Replicationcited by 76
  10. 2025OpenS2V-Nexus: A Detailed Benchmark and Million-Scale Dataset for Subject-to-Video Generationcited by 73
  11. 2024EchoMimicV2: Towards Striking, Simplified, and Semi-Body Human Animationcited by 69
  12. 2025Follow-Your-Creation: Empowering 4D Creation through Video Inpaintingcited by 61
38 more
  1. 2025Animate Anyone 2: High-Fidelity Character Image Animation with Environment Affordancecited by 60
  2. 2024IDOL: Instant Photorealistic 3D Human Creation from a Single Imagecited by 55
  3. 2024Diffusion Model-Based Video Editing: A Surveycited by 44
  4. 2025Follow-Your-Emoji-Faster: Towards Efficient, Fine-Controllable, and Expressive Freestyle Portrait Animationcited by 41
  5. 2024RealisDance: Equip controllable character animation with realistic handscited by 37
  6. 2025DreamActor-M1: Holistic, Expressive and Robust Human Image Animation with Hybrid Guidancecited by 36
  7. 2024DisPose: Disentangling Pose Guidance for Controllable Human Image Animationcited by 35
  8. 2025OmniHuman-1.5: Instilling an Active Mind in Avatars via Cognitive Simulationcited by 30
  9. 2025Lux Post Facto: Learning Portrait Performance Relighting with Conditional Video Diffusion and a Hybrid Datasetcited by 28
  10. 2024VividPose: Advancing Stable Video Diffusion for Realistic Human Image Animationcited by 27
  11. 2025X-NeMo: Expressive Neural Motion Reenactment via Disentangled Latent Attentioncited by 26
  12. 2025Human Motion Video Generation: A Surveycited by 25
  13. 2024A Comprehensive Survey on Human Video Generation: Challenges, Methods, and Insightscited by 24
  14. 2025GUAVA: Generalizable Upper Body 3D Gaussian Avatarcited by 22
  15. 2024GenLit: Reformulating Single-Image Relighting as Video Generationcited by 20
  16. 2024TALK-Act: Enhance Textural-Awareness for 2D Speaking Avatar Reenactment with Diffusion Modelcited by 19
  17. 2025GAS: Generative Avatar Synthesis from a Single Imagecited by 19
  18. 2025Re-HOLD: Video Hand Object Interaction Reenactment via adaptive Layout-instructed Diffusion Modelcited by 19
  19. 2025DreamActor-H1: High-Fidelity Human-Product Demonstration Video Generation via Motion-designed Diffusion Transformerscited by 19
  20. 2025X-Dyna: Expressive Dynamic Human Image Animationcited by 18
  21. 2024Joker: Conditional 3D Head Synthesis with Extreme Facial Expressionscited by 17
  22. 2024MotionStone: Decoupled Motion Intensity Modulation with Diffusion Transformer for Image-to-Video Generationcited by 16
  23. 2024PERSE: Personalized 3D Generative Avatars from A Single Portraitcited by 15
  24. 2025X-Dancer: Expressive Music to Human Dance Video Generationcited by 15
  25. 2025HunyuanVideo-HOMA: Generic Human-Object Interaction in Multimodal Driven Human Animationcited by 13
  26. 2025X-UniMotion: Animating Human Images with Expressive, Unified and Identity-Agnostic Motion Latentscited by 13
  27. 2024Buffer Anytime: Zero-Shot Video Depth and Normal from Image Priorscited by 11
  28. 2024RealisHuman: A Two-Stage Approach for Refining Malformed Human Parts in Generated Imagescited by 10
  29. 2025AudCast: Audio-Driven Human Video Generation by Cascaded Diffusion Transformerscited by 10
  30. 2025DiTaiListener: Controllable High Fidelity Listener Video Generation with Diffusioncited by 10
  31. 2025AniCrafter: Customizing Realistic Human-Centric Animation via Avatar-Background Conditioning in Video Diffusion Modelscited by 10
  32. 2025Animate-X++: Universal Character Image Animation with Dynamic Backgroundscited by 10
  33. 2024Dormant: Defending against Pose-driven Human Image Animationcited by 9
  34. 2025I2V3D: Controllable image-to-video generation with 3D guidancecited by 9
  35. 2025One-to-All Animation: Alignment-Free Character Animation and Image Pose Transfercited by 9
  36. 2024MikuDance: Animating Character Art with Mixed Motion Dynamicscited by 7
  37. 2025HumanGif: Single-View Human Diffusion with Generative Priorcited by 7
  38. 2025DynamiCtrl: Rethinking the Basic Structure and the Role of Text for High-quality Human Image Animationcited by 7

50 most cited of 115 All 115 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025MagicFace: High-Fidelity Facial Expression Editing with Action-Unit Control
  2. 2025FaceCrafter: Identity-Conditional Diffusion with Disentangled Control over Facial Pose, Expression, and Emotion
  3. 2024MagicFace: Training-free Universal-Style Human Image Customized Synthesis
  4. 2022Person Image Synthesis via Denoising Diffusion Model
  5. 2024Synthesizing Moving People with 3D Control
  6. 2024Towards a Simultaneous and Granular Identity-Expression Control in Personalized Face Generation
  7. 2025FantasyPortrait: Enhancing Multi-Character Portrait Animation with Expression-Augmented Diffusion Transformers
  8. 2025StableAnimator++: Overcoming Pose Misalignment and Face Distortion for Human Image Animation