# LivePortrait: Efficient Portrait Animation with Stitching and Retargeting Control

arXiv: https://arxiv.org/abs/2407.03168 · HTML: https://mlsanity.com/paper/2407.03168/

## Key facts

- Cited by 206 papers in the map (data through 14 Oct 2026).
- Builds on 48 of its 57 references that are in the map.
- First posted on arXiv 3 Jul 2024.

## Builds on (most cited first, 30 of 48)

- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [A Style-Based Generator Architecture for Generative Adversarial Networks](https://mlsanity.com/paper/1812.04948/) (2018)
- [Robust Speech Recognition via Large-Scale Weak Supervision](https://mlsanity.com/paper/2212.04356/) (2022)
- [Semantic Image Synthesis with Spatially-Adaptive Normalization](https://mlsanity.com/paper/1903.07291/) (2019)
- [Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network](https://mlsanity.com/paper/1609.05158/) (2016)
- [VoxCeleb: a large-scale speaker identification dataset](https://mlsanity.com/paper/1706.08612/) (2017)
- [ConvNeXt V2: Co-designing and Scaling ConvNets with Masked Autoencoders](https://mlsanity.com/paper/2301.00808/) (2023)
- [First Order Motion Model for Image Animation](https://mlsanity.com/paper/2003.00196/) (2020)
- [Animate Anyone: Consistent and Controllable Image-to-Video Synthesis for Character Animation](https://mlsanity.com/paper/2311.17117/) (2023)
- [One-Shot Free-View Neural Talking-Head Synthesis for Video Conferencing](https://mlsanity.com/paper/2011.15126/) (2020)
- [MagicAnimate: Temporally Consistent Human Image Animation using Diffusion Model](https://mlsanity.com/paper/2311.16498/) (2023)
- [StyleRig: Rigging StyleGAN for 3D Control over Portrait Images](https://mlsanity.com/paper/2004.00121/) (2020)
- [Motion Representations for Articulated Animation](https://mlsanity.com/paper/2104.11280/) (2021)
- [EMO: Emote Portrait Alive -- Generating Expressive Portrait Videos with Audio2Video Diffusion Model under Weak Conditions](https://mlsanity.com/paper/2402.17485/) (2024)
- [Thin-Plate Spline Motion Model for Image Animation](https://mlsanity.com/paper/2203.14367/) (2022)
- [FaceFormer: Speech-Driven 3D Facial Animation with Transformers](https://mlsanity.com/paper/2112.05329/) (2021)
- [NeRSemble: Multi-view Radiance Field Reconstruction of Human Heads](https://mlsanity.com/paper/2305.03027/) (2023)
- [AniPortrait: Audio-Driven Synthesis of Photorealistic Portrait Animation](https://mlsanity.com/paper/2403.17694/) (2024)
- [VASA-1: Lifelike Audio-Driven Talking Faces Generated in Real Time](https://mlsanity.com/paper/2404.10667/) (2024)
- [Depth-Aware Generative Adversarial Network for Talking Head Video Generation](https://mlsanity.com/paper/2203.06605/) (2022)
- [VFHQ: A High-Quality Dataset and Benchmark for Video Face Super-Resolution](https://mlsanity.com/paper/2205.03409/) (2022)
- [Follow-Your-Emoji: Fine-Controllable and Expressive Freestyle Portrait Animation](https://mlsanity.com/paper/2406.01900/) (2024)
- [DreamPose: Fashion Image-to-Video Synthesis via Stable Diffusion](https://mlsanity.com/paper/2304.06025/) (2023)
- [DisCo: Disentangled Control for Realistic Human Dance Generation](https://mlsanity.com/paper/2307.00040/) (2023)
- [DiffTalk: Crafting Diffusion Models for Generalized Audio-Driven Portraits Animation](https://mlsanity.com/paper/2301.03786/) (2023)
- [MegaPortraits: One-shot Megapixel Neural Head Avatars](https://mlsanity.com/paper/2207.07621/) (2022)
- [Realistic One-shot Mesh-based Head Avatars](https://mlsanity.com/paper/2206.08343/) (2022)

## Built on by (most cited first, 30 of 206)

- [Hallo3: Highly Dynamic and Realistic Portrait Image Animation with Video Diffusion Transformer](https://mlsanity.com/paper/2412.00733/) (2024)
- [StableAnimator: High-Quality Identity-Preserving Human Image Animation](https://mlsanity.com/paper/2411.17697/) (2024)
- [OmniAvatar: Efficient Audio-Driven Avatar Video Generation with Adaptive Body Animation](https://mlsanity.com/paper/2506.18866/) (2025)
- [Wan-Animate: Unified Character Animation and Replacement with Holistic Replication](https://mlsanity.com/paper/2509.14055/) (2025)
- [HunyuanPortrait: Implicit Condition Control for Enhanced Portrait Animation](https://mlsanity.com/paper/2503.18860/) (2025)
- [Long-Term TalkingFace Generation via Motion-Prior Conditional Diffusion Model](https://mlsanity.com/paper/2502.09533/) (2025)
- [LAM: Large Avatar Model for One-shot Animatable Gaussian Head](https://mlsanity.com/paper/2502.17796/) (2025)
- [Ditto: Motion-Space Diffusion for Controllable Realtime Talking Head Synthesis](https://mlsanity.com/paper/2411.19509/) (2024)
- [DreamActor-M1: Holistic, Expressive and Robust Human Image Animation with Hybrid Guidance](https://mlsanity.com/paper/2504.01724/) (2025)
- [FireRed-Image-Edit-1.0 Technical Report](https://mlsanity.com/paper/2602.13344/) (2026)
- [SkyReels-A1: Expressive Portrait Animation in Video Diffusion Transformers](https://mlsanity.com/paper/2502.10841/) (2025)
- [MuseTalk: Real-Time High-Fidelity Video Dubbing via Spatio-Temporal Sampling](https://mlsanity.com/paper/2410.10122/) (2024)
- [X-NeMo: Expressive Neural Motion Reenactment via Disentangled Latent Attention](https://mlsanity.com/paper/2507.23143/) (2025)
- [Human Motion Video Generation: A Survey](https://mlsanity.com/paper/2509.03883/) (2025)
- [Celeb-DF++: A Large-scale Challenging Video DeepFake Benchmark for Generalizable Forensics](https://mlsanity.com/paper/2507.18015/) (2025)
- [Kling-Avatar: Grounding Multimodal Instructions for Cascaded Long-Duration Avatar Animation Synthesis](https://mlsanity.com/paper/2509.09595/) (2025)
- [AnyTalker: Scaling Multi-Person Talking Video Generation with Interactivity Refinement](https://mlsanity.com/paper/2511.23475/) (2025)
- [Re-HOLD: Video Hand Object Interaction Reenactment via adaptive Layout-instructed Diffusion Model](https://mlsanity.com/paper/2503.16942/) (2025)
- [MIDAS: Multimodal Interactive Digital-humAn Synthesis via Real-time Autoregressive Video Generation](https://mlsanity.com/paper/2508.19320/) (2025)
- [Avatar Forcing: Real-Time Interactive Head Avatar Generation for Natural Conversation](https://mlsanity.com/paper/2601.00664/) (2026)
- [DreamDance: Animating Human Images by Enriching 3D Geometry Cues from 2D Poses](https://mlsanity.com/paper/2412.00397/) (2024)
- [Audio-visual Controlled Video Diffusion with Masked Selective State Spaces Modeling for Natural Talking Head Generation](https://mlsanity.com/paper/2504.02542/) (2025)
- [MVP4D: Multi-View Portrait Video Diffusion for Animatable 4D Avatars](https://mlsanity.com/paper/2510.12785/) (2025)
- [Teller: Real-Time Streaming Audio-Driven Portrait Animation with Autoregressive Motion Generation](https://mlsanity.com/paper/2503.18429/) (2025)
- [Hallo4: High-Fidelity Dynamic Portrait Animation via Direct Preference Optimization](https://mlsanity.com/paper/2505.23525/) (2025)
- [ARIG: Autoregressive Interactive Head Generation for Real-time Conversations](https://mlsanity.com/paper/2507.00472/) (2025)
- [PERSE: Personalized 3D Generative Avatars from A Single Portrait](https://mlsanity.com/paper/2412.21206/) (2024)
- [AvatarArtist: Open-Domain 4D Avatarization](https://mlsanity.com/paper/2503.19906/) (2025)
- [MVPortrait: Text-Guided Motion and Emotion Control for Multi-view Vivid Portrait Animation](https://mlsanity.com/paper/2503.19383/) (2025)
- [FixTalk: Taming Identity Leakage for High-Quality Talking Head Generation in Extreme Cases](https://mlsanity.com/paper/2507.01390/) (2025)

## Abstract (arXiv)

Portrait Animation aims to synthesize a lifelike video from a single source image, using it as an appearance reference, with motion (i.e., facial expressions and head pose) derived from a driving video, audio, text, or generation. Instead of following mainstream diffusion-based methods, we explore and extend the potential of the implicit-keypoint-based framework, which effectively balances computational efficiency and controllability. Building upon this, we develop a video-driven portrait animation framework named LivePortrait with a focus on better generalization, controllability, and efficiency for practical usage. To enhance the generation quality and generalization ability, we scale up the training data to about 69 million high-quality frames, adopt a mixed image-video training strategy, upgrade the network architecture, and design better motion transformation and optimization objectives. Additionally, we discover that compact implicit keypoints can effectively represent a kind of blendshapes and meticulously propose a stitching and two retargeting modules, which utilize a small MLP with negligible computational overhead, to enhance the controllability. Experimental results demonstrate the efficacy of our framework even compared to diffusion-based methods. The generation speed remarkably reaches 12.8ms on an RTX 4090 GPU with PyTorch. The inference code and models are available at https://github.com/KwaiVGI/LivePortrait