mlsanityData to 7 Oct 2026

HunyuanPortrait: Implicit Condition Control for Enhanced Portrait Animation

arXiv 2503.18860 ↗First posted 24 Mar 2025cs.CVcited by 55

Not on the map. Why?

Abstract (arXiv)

We introduce HunyuanPortrait, a diffusion-based condition control method that employs implicit representations for highly controllable and lifelike portrait animation. Given a single portrait image as an appearance reference and video clips as driving templates, HunyuanPortrait can animate the character in the reference image by the facial expression and head pose of the driving videos. In our framework, we utilize pre-trained encoders to achieve the decoupling of portrait motion information and identity in videos. To do so, implicit representation is adopted to encode motion information and is employed as control signals in the animation phase. By leveraging the power of stable video diffusion as the main building block, we carefully design adapter layers to inject control signals into the denoising unet through attention mechanisms. These bring spatial richness of details and temporal consistency. HunyuanPortrait also exhibits strong generalization performance, which can effectively disentangle appearance and motion under different image styles. Our framework outperforms existing methods, demonstrating superior temporal consistency and controllability. Our project is available at https://kkakkkka.github.io/HunyuanPortrait.

Key facts. Cited by 55 papers in the map (data through 7 Oct 2026). Builds on 55 of its 67 references that are in the map. First posted on arXiv 24 Mar 2025.

Builds on

55 of 67 references.

References · 55

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  4. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  5. 2020Denoising Diffusion Implicit Modelscited by 9,655
  6. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,764
  7. 2022Classifier-Free Diffusion Guidancecited by 5,717
  8. 2022Generative Adversarial Networkscited by 5,603
  9. 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasetscited by 2,526
  10. 2024HunyuanVideo: A Systematic Framework For Large Video Generative Modelscited by 1,597
  11. 2023IP-Adapter: Text Compatible Image Prompt Adapter for Text-to-Image Diffusion Modelscited by 1,439
  12. 2023AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuningcited by 1,418
38 more
  1. 2017Sigmoid-Weighted Linear Units for Neural Network Function Approximation in Reinforcement Learningcited by 935
  2. 2018VoxCeleb2: Deep Speaker Recognitioncited by 862
  3. 2020First Order Motion Model for Image Animationcited by 698
  4. 2023Animate Anyone: Consistent and Controllable Image-to-Video Synthesis for Character Animationcited by 683
  5. 2019MediaPipe: A Framework for Building Perception Pipelinescited by 572
  6. 2019FSGAN: Subject Agnostic Face Swapping and Reenactmentcited by 319
  7. 2023Follow Your Pose: Pose-Guided Text-to-Video Generation using Pose-Free Videoscited by 312
  8. 2023Effective Whole-body Pose Estimation with Two-stages Distillationcited by 307
  9. 2022CelebV-HQ: A Large-Scale Video Facial Attributes Datasetcited by 246
  10. 2019Understanding and Improving Layer Normalizationcited by 213
  11. 2022Thin-Plate Spline Motion Model for Image Animationcited by 209
  12. 2024LivePortrait: Efficient Portrait Animation with Stitching and Retargeting Controlcited by 202
  13. 2024MimicMotion: High-Quality Human Motion Video Generation with Confidence-aware Pose Guidancecited by 196
  14. 2024EchoMimic: Lifelike Audio-Driven Portrait Animations through Editable Landmark Conditionscited by 196
  15. 2024AniPortrait: Audio-Driven Synthesis of Photorealistic Portrait Animationcited by 191
  16. 2024VASA-1: Lifelike Audio-Driven Talking Faces Generated in Real Timecited by 186
  17. 2022VFHQ: A High-Quality Dataset and Benchmark for Video Face Super-Resolutioncited by 174
  18. 2024Follow-Your-Emoji: Fine-Controllable and Expressive Freestyle Portrait Animationcited by 172
  19. 2022StyleHEAT: One-Shot High-Resolution Editable Talking Face Generation via Pre-trained StyleGANcited by 169
  20. 2024ControlNeXt: Powerful and Efficient Control for Image and Video Generationcited by 158
  21. 2022MegaPortraits: One-shot Megapixel Neural Head Avatarscited by 131
  22. 2022Realistic One-shot Mesh-based Head Avatarscited by 119
  23. 2022VideoReTalking: Audio-based Lip Synchronization for Talking Head Video Editing In the Wildcited by 106
  24. 2024X-Portrait: Expressive Portrait Animation with Hierarchical Motion Attentioncited by 88
  25. 2021One-shot Talking Face Generation from Single-speaker Audio-Visual Correlation Learningcited by 87
  26. 2022FineDance: A Fine-grained Choreography Dataset for 3D Full Body Dance Generationcited by 87
  27. 2024V-Express: Conditional Dropout for Progressive Training of Portrait Video Generationcited by 85
  28. 2024SMIRK: 3D Facial Expressions through Analysis-by-Neural-Synthesiscited by 81
  29. 2022MetaPortrait: Identity-Preserving Talking Head Generation with Fast Personalized Adaptationcited by 69
  30. 2023Implicit Identity Representation Conditioned Memory Compensation Network for Talking Head video Generationcited by 61
  31. 2024Lodge: A Coarse to Fine Diffusion Network for Long Dance Generation Guided by the Characteristic Dance Primitivescited by 59
  32. 2024MambaTalk: Efficient Holistic Gesture Synthesis with Selective State Space Modelscited by 46
  33. 2024MEMO: Memory-Guided Diffusion for Expressive Talking Video Generationcited by 38
  34. 2024DisPose: Disentangling Pose Guidance for Controllable Human Image Animationcited by 35
  35. 2024MegActor: Harness the Power of Raw Video for Vivid Portrait Animationcited by 27
  36. 2023DiffPortrait3D: Controllable Diffusion for Zero-Shot Portrait View Synthesiscited by 26
  37. 2024Privacy-Preserving Face Recognition Using Trainable Feature Subtractioncited by 24
  38. 2023G2L: Semantically Aligned and Uniform Video Grounding via Geodesic and Game Theorycited by 21

50 most cited of 55 All 55 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 29; 2026: 26.

55 papers.

Sort:

By area:

  1. 2025Controllable Video Generation: A Surveycited by 93
  2. 2025HunyuanCustom: A Multimodal-Driven Architecture for Customized Video Generationcited by 84
  3. 2025Wan-Animate: Unified Character Animation and Replacement with Holistic Replicationcited by 76
  4. 2025Follow-Your-Emoji-Faster: Towards Efficient, Fine-Controllable, and Expressive Freestyle Portrait Animationcited by 41
  5. 2025Adversarial Distribution Matching for Diffusion Distillation Towards Efficient Image and Video Synthesiscited by 39
  6. 2025FireEdit: Fine-grained Instruction-based Image Editing via Region-aware Vision Language Modelcited by 31
  7. 2025MindOmni: Unleashing Reasoning Generation in Vision Language Models with RGPOcited by 30
  8. 2025Omni-Effects: Unified and Spatially-Controllable Visual Effects Generationcited by 27
  9. 2025OmniV2V: Versatile Video Generation and Editing via Dynamic Content Manipulationcited by 18
  10. 2025Audio-visual Controlled Video Diffusion with Masked Selective State Spaces Modeling for Natural Talking Head Generationcited by 17
  11. 2026Avatar Forcing: Real-Time Interactive Head Avatar Generation for Natural Conversationcited by 17
  12. 2025MVPortrait: Text-Guided Motion and Emotion Control for Multi-view Vivid Portrait Animationcited by 14
38 more
  1. 2025FantasyPortrait: Enhancing Multi-Character Portrait Animation with Expression-Augmented Diffusion Transformerscited by 12
  2. 2025SEGA: Drivable 3D Gaussian Head Avatar from a Single Imagecited by 11
  3. 2025MS-DETR: Towards Effective Video Moment Retrieval and Highlight Detection by Joint Motion-Semantic Learningcited by 10
  4. 2025MIND-V: Hierarchical World Model for Long-Horizon Robotic Manipulation with RL-based Physical Alignmentcited by 9
  5. 2025Fine-Grained Zero-Shot Object Detectioncited by 8
  6. 2025Zo3T: Zero-Shot 3D-Aware Trajectory-Guided Image-to-Video Generation via Test-Time Trainingcited by 7
  7. 2025Advancing Talking Head Generation: A Comprehensive Survey of Multi-Modal Methodologies, Datasets, Evaluation Metrics, and Loss Functionscited by 6
  8. 2025PersonaLive! Expressive Portrait Image Animation for Live Streamingcited by 6
  9. 2025ViBES: A Conversational Agent with Behaviorally-Intelligent 3D Virtual Bodycited by 6
  10. 2025Separate to Collaborate: Dual-Stream Diffusion Model for Coordinated Piano Hand Motion Synthesiscited by 4
  11. 2025InterAnimate: Taming Region-aware Diffusion Model for Realistic Human Interaction Animationcited by 4
  12. 2025MultiMotion: Multi Subject Video Motion Transfer via Video Diffusion Transformercited by 3
  13. 2025FlashPortrait: 6x Faster Infinite Portrait Animation with Adaptive Latent Predictioncited by 3
  14. 2026MotionWeaver: Holistic 4D-Anchored Framework for Multi-Humanoid Image Animationcited by 3
  15. 2026Identity-Consistent Video Generation under Large Facial-Angle Variationscited by 3
  16. 2025MTVCraft: Tokenizing 4D Motion for Arbitrary Character Animationcited by 2
  17. 2026EditYourself: Audio-Driven Generation and Manipulation of Talking Head Videos with Diffusion Transformerscited by 2
  18. 2026AHOY! Animatable Humans under Occlusion from YouTube Videos with Gaussian Splatting and Video Diffusion Priorscited by 2
  19. 2026Leveraging Avatar Fingerprinting: A Multi-Generator Photorealistic Talking-Head Public Database and Benchmarkcited by 2
  20. 2026CogPortrait: Fine-Grained Eye-Region Control in Portrait Animation via Hierarchical Agent Planningcited by 2
  21. 2026InteracVid: Building a Real Interactive Audio-Visual Response Dataset from Live-Chat Videoscited by 2
  22. 2025Generative Augmented Reality: Paradigms, Technologies, and Future Applicationscited by 1
  23. 2025FactorPortrait: Controllable Portrait Animation via Disentangled Expression, Pose, and Viewpointcited by 1
  24. 2026IM-Animation: An Implicit Motion Representation for Identity-decoupled Character Animationcited by 1
  25. 2026AUHead: Realistic Emotional Talking Head Generation via Action Units Controlcited by 1
  26. 2026Image-to-Video Diffusion: From Foundations to Open Frontierscited by 1
  27. 2026VidForensics-M1: Meta-Detection Reinforcement Learning with Verifiable Temporal Grounding for AI-Generated Video Forensicscited by 1
  28. 2025FaceEditTalker: Controllable Talking Head Generation with Facial Attribute Editingcited by 0
  29. 2025Instant Expressive Gaussian Head Avatars at Over 100 FPScited by 0
  30. 2026ExpPortrait: Expressive Portrait Generation via Personalized Representationcited by 0
  31. 2026LiftAvatar: Kinematic-Space Completion for Expression-Controlled 3D Gaussian Avatar Animationcited by 0
  32. 2026UniBioTransfer: A Unified Framework for Multiple Biometrics Transfercited by 0
  33. 2026PerformRecast: Expression and Head Pose Disentanglement for Portrait Video Editingcited by 0
  34. 2026FG-Portrait: 3D Flow Guided Editable Portrait Animationcited by 0
  35. 2026PortraitDirector: A Hierarchical Disentanglement Framework for Controllable and Real-time Facial Reenactmentcited by 0
  36. 2026Loki: Representation over Architecture for Diffusion-Based Portrait Animationcited by 0
  37. 2026Resonant Minds: Closed-Loop Social Avatars with Theory of Mindcited by 0
  38. 2026fMRI2Face: A Full-HD fMRI-Video Dataset and Geometry-Guided Neural Decoding Framework for Dynamic Human Face Reconstructioncited by 0

50 most cited of 55 All 55 (JSON)

Similar, not cited

Close in wording, no citation either way

  1. 2025Stable Video-Driven Portraits
  2. 2025DeX-Portrait: Disentangled and Expressive Portrait Animation via Explicit and Latent Motion Representations
  3. 2025High-Fidelity Relightable Monocular Portrait Animation with Lighting-Controllable Video Diffusion Model
  4. 2024Hallo3: Highly Dynamic and Realistic Portrait Image Animation with Video Diffusion Transformer
  5. 2024AniFaceDiff: Animating Stylized Avatars via Parametric Conditioned Diffusion Models
  6. 2025DynamiCtrl: Rethinking the Basic Structure and the Role of Text for High-quality Human Image Animation
  7. 2024MegActor-Sigma: Unlocking Flexible Mixed-Modal Control in Portrait Animation with Diffusion Transformer
  8. 2025FramePrompt: In-context Controllable Animation with Zero Structural Changes