mlsanityData to 7 Oct 2026

LivePortrait: Efficient Portrait Animation with Stitching and Retargeting Control

arXiv 2407.03168 ↗First posted 3 Jul 2024cs.CVcited by 202

Not on the map. Why?

Cited by 51 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Portrait Animation aims to synthesize a lifelike video from a single source image, using it as an appearance reference, with motion (i.e., facial expressions and head pose) derived from a driving video, audio, text, or generation. Instead of following mainstream diffusion-based methods, we explore and extend the potential of the implicit-keypoint-based framework, which effectively balances computational efficiency and controllability. Building upon this, we develop a video-driven portrait animation framework named LivePortrait with a focus on better generalization, controllability, and efficiency for practical usage. To enhance the generation quality and generalization ability, we scale up the training data to about 69 million high-quality frames, adopt a mixed image-video training strategy, upgrade the network architecture, and design better motion transformation and optimization objectives. Additionally, we discover that compact implicit keypoints can effectively represent a kind of blendshapes and meticulously propose a stitching and two retargeting modules, which utilize a small MLP with negligible computational overhead, to enhance the controllability. Experimental results demonstrate the efficacy of our framework even compared to diffusion-based methods. The generation speed remarkably reaches 12.8ms on an RTX 4090 GPU with PyTorch. The inference code and models are available at https://github.com/KwaiVGI/LivePortrait

Key facts. Cited by 202 papers in the map (data through 7 Oct 2026). Builds on 48 of its 57 references that are in the map. First posted on arXiv 3 Jul 2024.

Builds on

48 of 57 references.

References · 48

  1. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  3. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  4. 2020Denoising Diffusion Implicit Modelscited by 9,655
  5. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
  6. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 3,972
  7. 2019Semantic Image Synthesis with Spatially-Adaptive Normalizationcited by 1,295
  8. 2016Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Networkcited by 1,199
  9. 2017VoxCeleb: a large-scale speaker identification datasetcited by 714
  10. 2023ConvNeXt V2: Co-designing and Scaling ConvNets with Masked Autoencoderscited by 703
  11. 2020First Order Motion Model for Image Animationcited by 698
  12. 2023Animate Anyone: Consistent and Controllable Image-to-Video Synthesis for Character Animationcited by 683
36 more
  1. 2020One-Shot Free-View Neural Talking-Head Synthesis for Video Conferencingcited by 438
  2. 2023MagicAnimate: Temporally Consistent Human Image Animation using Diffusion Modelcited by 330
  3. 2020StyleRig: Rigging StyleGAN for 3D Control over Portrait Imagescited by 267
  4. 2021Motion Representations for Articulated Animationcited by 253
  5. 2024EMO: Emote Portrait Alive -- Generating Expressive Portrait Videos with Audio2Video Diffusion Model under Weak Conditionscited by 233
  6. 2022Thin-Plate Spline Motion Model for Image Animationcited by 209
  7. 2021FaceFormer: Speech-Driven 3D Facial Animation with Transformerscited by 208
  8. 2024AniPortrait: Audio-Driven Synthesis of Photorealistic Portrait Animationcited by 191
  9. 2023NeRSemble: Multi-view Radiance Field Reconstruction of Human Headscited by 190
  10. 2024VASA-1: Lifelike Audio-Driven Talking Faces Generated in Real Timecited by 186
  11. 2022Depth-Aware Generative Adversarial Network for Talking Head Video Generationcited by 177
  12. 2022VFHQ: A High-Quality Dataset and Benchmark for Video Face Super-Resolutioncited by 174
  13. 2024Follow-Your-Emoji: Fine-Controllable and Expressive Freestyle Portrait Animationcited by 172
  14. 2023DreamPose: Fashion Image-to-Video Synthesis via Stable Diffusioncited by 167
  15. 2023DisCo: Disentangled Control for Realistic Human Dance Generationcited by 141
  16. 2023DiffTalk: Crafting Diffusion Models for Generalized Audio-Driven Portraits Animationcited by 136
  17. 2022MegaPortraits: One-shot Megapixel Neural Head Avatarscited by 131
  18. 2022Realistic One-shot Mesh-based Head Avatarscited by 119
  19. 2023MagicPose: Realistic Human Poses and Facial Expressions Retargeting with Identity-aware Diffusioncited by 115
  20. 2017Wing Loss for Robust Facial Landmark Localisation with Convolutional Neural Networkscited by 108
  21. 2024X-Portrait: Expressive Portrait Animation with Hierarchical Motion Attentioncited by 88
  22. 2024V-Express: Conditional Dropout for Progressive Training of Portrait Video Generationcited by 85
  23. 2024SMIRK: 3D Facial Expressions through Analysis-by-Neural-Synthesiscited by 81
  24. 2024AniTalker: Animate Vivid and Diverse Talking Faces through Identity-Decoupled Facial Motion Encodingcited by 67
  25. 2024Face Adapter for Pre-Trained Diffusion Models with Fine-Grained ID and Attribute Controlcited by 66
  26. 2022L2CS-Net: Fine-Grained Gaze Estimation in Unconstrained Environmentscited by 65
  27. 2024EMOPortraits: Emotion-enhanced Multimodal One-shot Head Avatarscited by 65
  28. 2023Implicit Identity Representation Conditioned Memory Compensation Network for Talking Head video Generationcited by 61
  29. 2020GIF: Generative Interpretable Facescited by 53
  30. 2023Face Animation with an Attribute-Guided Diffusion Modelcited by 48
  31. 2021BlendGAN: Implicitly GAN Blending for Arbitrary Stylized Face Generationcited by 47
  32. 2022Implicit Warping for Animation with Image Setscited by 45
  33. 2023Towards Practical Capture of High-Fidelity Relightable Avatarscited by 34
  34. 2024MegActor: Harness the Power of Raw Video for Vivid Portrait Animationcited by 27
  35. 2024VRMM: A Volumetric Relightable Morphable Head Modelcited by 19
  36. 2023DiffTalker: Co-driven audio-image diffusion for talking faces via intermediate landmarkscited by 9

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 20; 2025: 116; 2026: 66.

202 papers.

Sort:

By area:

  1. 2024Hallo3: Highly Dynamic and Realistic Portrait Image Animation with Video Diffusion Transformercited by 121
  2. 2024StableAnimator: High-Quality Identity-Preserving Human Image Animationcited by 85
  3. 2025OmniAvatar: Efficient Audio-Driven Avatar Video Generation with Adaptive Body Animationcited by 77
  4. 2025Wan-Animate: Unified Character Animation and Replacement with Holistic Replicationcited by 76
  5. 2025HunyuanPortrait: Implicit Condition Control for Enhanced Portrait Animationcited by 55
  6. 2025Long-Term TalkingFace Generation via Motion-Prior Conditional Diffusion Modelcited by 49
  7. 2025LAM: Large Avatar Model for One-shot Animatable Gaussian Headcited by 43
  8. 2024Ditto: Motion-Space Diffusion for Controllable Realtime Talking Head Synthesiscited by 41
  9. 2025DreamActor-M1: Holistic, Expressive and Robust Human Image Animation with Hybrid Guidancecited by 36
  10. 2025SkyReels-A1: Expressive Portrait Animation in Video Diffusion Transformerscited by 33
  11. 2026FireRed-Image-Edit-1.0 Technical Reportcited by 31
  12. 2024MuseTalk: Real-Time High-Fidelity Video Dubbing via Spatio-Temporal Samplingcited by 27
38 more
  1. 2025X-NeMo: Expressive Neural Motion Reenactment via Disentangled Latent Attentioncited by 26
  2. 2025Human Motion Video Generation: A Surveycited by 25
  3. 2025Kling-Avatar: Grounding Multimodal Instructions for Cascaded Long-Duration Avatar Animation Synthesiscited by 20
  4. 2025Re-HOLD: Video Hand Object Interaction Reenactment via adaptive Layout-instructed Diffusion Modelcited by 19
  5. 2025AnyTalker: Scaling Multi-Person Talking Video Generation with Interactivity Refinementcited by 19
  6. 2025Celeb-DF++: A Large-scale Challenging Video DeepFake Benchmark for Generalizable Forensicscited by 18
  7. 2025MIDAS: Multimodal Interactive Digital-humAn Synthesis via Real-time Autoregressive Video Generationcited by 18
  8. 2025Audio-visual Controlled Video Diffusion with Masked Selective State Spaces Modeling for Natural Talking Head Generationcited by 17
  9. 2025MVP4D: Multi-View Portrait Video Diffusion for Animatable 4D Avatarscited by 17
  10. 2026Avatar Forcing: Real-Time Interactive Head Avatar Generation for Natural Conversationcited by 17
  11. 2024DreamDance: Animating Human Images by Enriching 3D Geometry Cues from 2D Posescited by 16
  12. 2025Teller: Real-Time Streaming Audio-Driven Portrait Animation with Autoregressive Motion Generationcited by 16
  13. 2025Hallo4: High-Fidelity Dynamic Portrait Animation via Direct Preference Optimizationcited by 16
  14. 2025ARIG: Autoregressive Interactive Head Generation for Real-time Conversationscited by 16
  15. 2024PERSE: Personalized 3D Generative Avatars from A Single Portraitcited by 15
  16. 2025AvatarArtist: Open-Domain 4D Avatarizationcited by 15
  17. 2025MVPortrait: Text-Guided Motion and Emotion Control for Multi-view Vivid Portrait Animationcited by 14
  18. 2025FixTalk: Taming Identity Leakage for High-Quality Talking Head Generation in Extreme Casescited by 14
  19. 2024MegActor-Sigma: Unlocking Flexible Mixed-Modal Control in Portrait Animation with Diffusion Transformercited by 13
  20. 2025PVChat: Personalized Video Chat with One-Shot Learningcited by 13
  21. 2025TransPixeler: Advancing Text-to-Video Generation with Transparencycited by 12
  22. 2025FaceShot: Bring Any Character into Lifecited by 12
  23. 2025Cafe-Talk: Generating 3D Talking Face Animation with Multimodal Coarse- and Fine-grained Controlcited by 12
  24. 2025FantasyPortrait: Enhancing Multi-Character Portrait Animation with Expression-Augmented Diffusion Transformerscited by 12
  25. 2025StableAnimator++: Overcoming Pose Misalignment and Face Distortion for Human Image Animationcited by 12
  26. 2025SOAP: Style-Omniscient Animatable Portraitscited by 11
  27. 2025KlingAvatar 2.0 Technical Reportcited by 11
  28. 2025Knot Forcing: Taming Autoregressive Video Diffusion Models for Real-time Infinite Interactive Portrait Animationcited by 11
  29. 2024GPHM: Gaussian Parametric Head Model for Monocular Head Avatar Reconstructioncited by 10
  30. 2025TalkingHeadBench: A Multi-Modal Benchmark & Analysis of Talking-Head DeepFake Detectioncited by 10
  31. 2025SoulX-FlashTalk: Real-Time Infinite Streaming of Audio-Driven Avatars via Self-Correcting Bidirectional Distillationcited by 10
  32. 2024JoyVASA: Portrait and Animal Image Animation with Diffusion-Based Audio-Driven Facial Dynamics and Head Motion Generationcited by 9
  33. 2025OmniTalker: One-shot Real-time Text-Driven Talking Audio-Video Generation With Multimodal Style Mimickingcited by 9
  34. 2025MAVOS-DD: Multilingual Audio-Video Open-Set Deepfake Detection Benchmarkcited by 9
  35. 2025SyncTalk++: High-Fidelity and Efficient Synchronized Talking Heads Synthesis Using Gaussian Splattingcited by 9
  36. 2025One-to-All Animation: Alignment-Free Character Animation and Image Pose Transfercited by 9
  37. 2025Scaling Zero-Shot Reference-to-Video Generationcited by 9
  38. 2024DH-FaceVid-1K: A Large-Scale High-Quality Dataset for Face Video Generationcited by 8

50 most cited of 202 All 202 (JSON)