# HunyuanPortrait: Implicit Condition Control for Enhanced Portrait Animation

arXiv: https://arxiv.org/abs/2503.18860 · HTML: https://mlsanity.com/paper/2503.18860/

## Key facts

- Cited by 59 papers in the map (data through 14 Oct 2026).
- Builds on 55 of its 67 references that are in the map.
- First posted on arXiv 24 Mar 2025.

## Builds on (most cited first, 30 of 55)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [Classifier-Free Diffusion Guidance](https://mlsanity.com/paper/2207.12598/) (2022)
- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasets](https://mlsanity.com/paper/2311.15127/) (2023)
- [HunyuanVideo: A Systematic Framework For Large Video Generative Models](https://mlsanity.com/paper/2412.03603/) (2024)
- [IP-Adapter: Text Compatible Image Prompt Adapter for Text-to-Image Diffusion Models](https://mlsanity.com/paper/2308.06721/) (2023)
- [AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuning](https://mlsanity.com/paper/2307.04725/) (2023)
- [Sigmoid-Weighted Linear Units for Neural Network Function Approximation in Reinforcement Learning](https://mlsanity.com/paper/1702.03118/) (2017)
- [VoxCeleb2: Deep Speaker Recognition](https://mlsanity.com/paper/1806.05622/) (2018)
- [First Order Motion Model for Image Animation](https://mlsanity.com/paper/2003.00196/) (2020)
- [Animate Anyone: Consistent and Controllable Image-to-Video Synthesis for Character Animation](https://mlsanity.com/paper/2311.17117/) (2023)
- [MediaPipe: A Framework for Building Perception Pipelines](https://mlsanity.com/paper/1906.08172/) (2019)
- [FSGAN: Subject Agnostic Face Swapping and Reenactment](https://mlsanity.com/paper/1908.05932/) (2019)
- [Follow Your Pose: Pose-Guided Text-to-Video Generation using Pose-Free Videos](https://mlsanity.com/paper/2304.01186/) (2023)
- [Effective Whole-body Pose Estimation with Two-stages Distillation](https://mlsanity.com/paper/2307.15880/) (2023)
- [CelebV-HQ: A Large-Scale Video Facial Attributes Dataset](https://mlsanity.com/paper/2207.12393/) (2022)
- [Understanding and Improving Layer Normalization](https://mlsanity.com/paper/1911.07013/) (2019)
- [Thin-Plate Spline Motion Model for Image Animation](https://mlsanity.com/paper/2203.14367/) (2022)
- [LivePortrait: Efficient Portrait Animation with Stitching and Retargeting Control](https://mlsanity.com/paper/2407.03168/) (2024)
- [MimicMotion: High-Quality Human Motion Video Generation with Confidence-aware Pose Guidance](https://mlsanity.com/paper/2406.19680/) (2024)
- [EchoMimic: Lifelike Audio-Driven Portrait Animations through Editable Landmark Conditions](https://mlsanity.com/paper/2407.08136/) (2024)
- [AniPortrait: Audio-Driven Synthesis of Photorealistic Portrait Animation](https://mlsanity.com/paper/2403.17694/) (2024)
- [VASA-1: Lifelike Audio-Driven Talking Faces Generated in Real Time](https://mlsanity.com/paper/2404.10667/) (2024)
- [VFHQ: A High-Quality Dataset and Benchmark for Video Face Super-Resolution](https://mlsanity.com/paper/2205.03409/) (2022)
- [Follow-Your-Emoji: Fine-Controllable and Expressive Freestyle Portrait Animation](https://mlsanity.com/paper/2406.01900/) (2024)

## Built on by (most cited first, 30 of 59)

- [Controllable Video Generation: A Survey](https://mlsanity.com/paper/2507.16869/) (2025)
- [HunyuanCustom: A Multimodal-Driven Architecture for Customized Video Generation](https://mlsanity.com/paper/2505.04512/) (2025)
- [Wan-Animate: Unified Character Animation and Replacement with Holistic Replication](https://mlsanity.com/paper/2509.14055/) (2025)
- [Follow-Your-Emoji-Faster: Towards Efficient, Fine-Controllable, and Expressive Freestyle Portrait Animation](https://mlsanity.com/paper/2509.16630/) (2025)
- [Adversarial Distribution Matching for Diffusion Distillation Towards Efficient Image and Video Synthesis](https://mlsanity.com/paper/2507.18569/) (2025)
- [FireEdit: Fine-grained Instruction-based Image Editing via Region-aware Vision Language Model](https://mlsanity.com/paper/2503.19839/) (2025)
- [MindOmni: Unleashing Reasoning Generation in Vision Language Models with RGPO](https://mlsanity.com/paper/2505.13031/) (2025)
- [Omni-Effects: Unified and Spatially-Controllable Visual Effects Generation](https://mlsanity.com/paper/2508.07981/) (2025)
- [OmniV2V: Versatile Video Generation and Editing via Dynamic Content Manipulation](https://mlsanity.com/paper/2506.01801/) (2025)
- [Avatar Forcing: Real-Time Interactive Head Avatar Generation for Natural Conversation](https://mlsanity.com/paper/2601.00664/) (2026)
- [Audio-visual Controlled Video Diffusion with Masked Selective State Spaces Modeling for Natural Talking Head Generation](https://mlsanity.com/paper/2504.02542/) (2025)
- [MVPortrait: Text-Guided Motion and Emotion Control for Multi-view Vivid Portrait Animation](https://mlsanity.com/paper/2503.19383/) (2025)
- [FantasyPortrait: Enhancing Multi-Character Portrait Animation with Expression-Augmented Diffusion Transformers](https://mlsanity.com/paper/2507.12956/) (2025)
- [SEGA: Drivable 3D Gaussian Head Avatar from a Single Image](https://mlsanity.com/paper/2504.14373/) (2025)
- [MS-DETR: Towards Effective Video Moment Retrieval and Highlight Detection by Joint Motion-Semantic Learning](https://mlsanity.com/paper/2507.12062/) (2025)
- [Zo3T: Zero-Shot 3D-Aware Trajectory-Guided Image-to-Video Generation via Test-Time Training](https://mlsanity.com/paper/2509.06723/) (2025)
- [MIND-V: Hierarchical World Model for Long-Horizon Robotic Manipulation with RL-based Physical Alignment](https://mlsanity.com/paper/2512.06628/) (2025)
- [PersonaLive! Expressive Portrait Image Animation for Live Streaming](https://mlsanity.com/paper/2512.11253/) (2025)
- [Fine-Grained Zero-Shot Object Detection](https://mlsanity.com/paper/2507.10358/) (2025)
- [Advancing Talking Head Generation: A Comprehensive Survey of Multi-Modal Methodologies, Datasets, Evaluation Metrics, and Loss Functions](https://mlsanity.com/paper/2507.02900/) (2025)
- [ViBES: A Conversational Agent with Behaviorally-Intelligent 3D Virtual Body](https://mlsanity.com/paper/2512.14234/) (2025)
- [FlashPortrait: 6x Faster Infinite Portrait Animation with Adaptive Latent Prediction](https://mlsanity.com/paper/2512.16900/) (2025)
- [Separate to Collaborate: Dual-Stream Diffusion Model for Coordinated Piano Hand Motion Synthesis](https://mlsanity.com/paper/2504.09885/) (2025)
- [InterAnimate: Taming Region-aware Diffusion Model for Realistic Human Interaction Animation](https://mlsanity.com/paper/2504.10905/) (2025)
- [MTVCraft: Tokenizing 4D Motion for Arbitrary Character Animation](https://mlsanity.com/paper/2505.10238/) (2025)
- [MultiMotion: Multi Subject Video Motion Transfer via Video Diffusion Transformer](https://mlsanity.com/paper/2512.07500/) (2025)
- [MotionWeaver: Holistic 4D-Anchored Framework for Multi-Humanoid Image Animation](https://mlsanity.com/paper/2602.13326/) (2026)
- [Identity-Consistent Video Generation under Large Facial-Angle Variations](https://mlsanity.com/paper/2603.21299/) (2026)
- [Image-to-Video Diffusion: From Foundations to Open Frontiers](https://mlsanity.com/paper/2605.17248/) (2026)
- [EditYourself: Audio-Driven Generation and Manipulation of Talking Head Videos with Diffusion Transformers](https://mlsanity.com/paper/2601.22127/) (2026)

## Abstract (arXiv)

We introduce HunyuanPortrait, a diffusion-based condition control method that employs implicit representations for highly controllable and lifelike portrait animation. Given a single portrait image as an appearance reference and video clips as driving templates, HunyuanPortrait can animate the character in the reference image by the facial expression and head pose of the driving videos. In our framework, we utilize pre-trained encoders to achieve the decoupling of portrait motion information and identity in videos. To do so, implicit representation is adopted to encode motion information and is employed as control signals in the animation phase. By leveraging the power of stable video diffusion as the main building block, we carefully design adapter layers to inject control signals into the denoising unet through attention mechanisms. These bring spatial richness of details and temporal consistency. HunyuanPortrait also exhibits strong generalization performance, which can effectively disentangle appearance and motion under different image styles. Our framework outperforms existing methods, demonstrating superior temporal consistency and controllability. Our project is available at https://kkakkkka.github.io/HunyuanPortrait.