mlsanityData to 14 Oct 2026

Long-Term TalkingFace Generation via Motion-Prior Conditional Diffusion Model

arXiv 2502.09533 ↗First posted 13 Feb 2025cs.CVcited by 49

On the map: Diffusion models → Audio, speech and music

Abstract (arXiv)

Recent advances in conditional diffusion models have shown promise for generating realistic TalkingFace videos, yet challenges persist in achieving consistent head movement, synchronized facial expressions, and accurate lip synchronization over extended generations. To address these, we introduce the Motion-priors Conditional Diffusion Model (MCDM), which utilizes both archived and current clip motion priors to enhance motion prediction and ensure temporal consistency. The model consists of three key elements: (1) an archived-clip motion-prior that incorporates historical frames and a reference frame to preserve identity and context; (2) a present-clip motion-prior diffusion model that captures multimodal causality for accurate predictions of head movements, lip sync, and expressions; and (3) a memory-efficient temporal attention mechanism that mitigates error accumulation by dynamically storing and updating motion features. We also release the TalkingFace-Wild dataset, a multilingual collection of over 200 hours of footage across 10 languages. Experimental results demonstrate the effectiveness of MCDM in maintaining identity and motion continuity for long-term TalkingFace generation. Code, models, and datasets will be publicly available.

Key facts. Cited by 49 papers in the map (data through 14 Oct 2026). Placed in Audio, speech and music, one of 14 directions in Diffusion models. Builds on 17 of its 54 references that are in the map. First posted on arXiv 13 Feb 2025.

Builds on

17 of 54 references.

Closest ancestors · 3

Same direction, newest first

  1. 2024Loopy: Taming Audio-Driven Portrait Avatar with Long-Term Motion Dependency4 Sep
  2. 2024MegActor-Sigma: Unlocking Flexible Mixed-Modal Control in Portrait Animation with Diffusion Transformer27 Aug
  3. 2024EMO: Emote Portrait Alive -- Generating Expressive Portrait Videos with Audio2Video Diffusion Model under Weak Conditions27 Feb

Other references · 14

  1. 2014Conditional Generative Adversarial Netscited by 2,162
  2. 2020Rethinking Attention with Performerscited by 1,601
  3. 2023AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuningcited by 1,433
  4. 2024LivePortrait: Efficient Portrait Animation with Stitching and Retargeting Controlcited by 206
  5. 2024Hallo: Hierarchical Audio-Driven Visual Synthesis for Portrait Image Animationcited by 202
  6. 2024EchoMimic: Lifelike Audio-Driven Portrait Animations through Editable Landmark Conditionscited by 196
  7. 2024AniPortrait: Audio-Driven Synthesis of Photorealistic Portrait Animationcited by 194
  8. 2024Follow-Your-Emoji: Fine-Controllable and Expressive Freestyle Portrait Animationcited by 174
  9. 2021Audio2Head: Audio-driven One-shot Talking-head Generation with Natural Head Motioncited by 149
  10. 2023GeneFace: Generalized and High-Fidelity Audio-Driven 3D Talking Face Synthesiscited by 131
  11. 2024Real3D-Portrait: One-shot Realistic 3D Talking Portrait Synthesiscited by 97
  12. 2021Sample and Computation Redistribution for Efficient Face Detectioncited by 89
2 more
  1. 2024V-Express: Conditional Dropout for Progressive Training of Portrait Video Generationcited by 86
  2. 2023VividTalk: One-Shot Audio-Driven Talking Head Generation Based on 3D Hybrid Priorcited by 47

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 31; 2026: 18.

49 papers.

Sort:

By area:

  1. 2025Controllable Video Generation: A Surveycited by 98
  2. 2025FaceShot: Bring Any Character into Lifecited by 13
  3. 2025Advanced Sign Language Video Generation with Compressed and Quantized Multi-Condition Tokenizationcited by 9
  4. 2025PSR: Scaling Multi-Subject Personalized Image Generation with Pairwise Subject-Consistency Rewardscited by 9
  5. 2026WildActor: Unconstrained Identity-Preserving Video Generationcited by 4
  6. 2025FreqU-FNet: Frequency-Aware U-Net for Imbalanced Medical Image Segmentationcited by 3
  7. 2025MultiMotion: Multi Subject Video Motion Transfer via Video Diffusion Transformercited by 3
  8. 2025HDM: Hybrid Diffusion Model for Unified Image Anomaly Detectioncited by 1
  9. 2025An improved EfficientNetV2 for garbage classificationcited by 1
  10. 2025FaceEditTalker: Controllable Talking Head Generation with Facial Attribute Editingcited by 1
  11. 2025MCFNet: A Multimodal Collaborative Fusion Network for Fine-Grained Semantic Classificationcited by 1
  12. 2025YOLO-FDA: Integrating Hierarchical Attention and Detail Enhancement for Surface Defect Detectioncited by 1
37 more
  1. 2025RefAdGen: High-Fidelity Advertising Image Generationcited by 1
  2. 2026AUHead: Realistic Emotional Talking Head Generation via Action Units Controlcited by 1
  3. 2026Normality-Preserving Continual Industrial Anomaly Detection via Orthogonal LoRA Bankscited by 1
  4. 20252DMCG:2DMambawith Change Flow Guidance for Change Detection in Remote Sensingcited by 0
  5. 2025CFMD: Dynamic Cross-layer Feature Fusion for Salient Object Detectioncited by 0
  6. 2025Photovoltaic Defect Image Generator with Boundary Alignment Smoothing Constraint for Domain Shift Mitigationcited by 0
  7. 2025Dual Attention Residual U-Net for Accurate Brain Ultrasound Segmentation in IVH Detectioncited by 0
  8. 2025YOLO-SPCI: Enhancing Remote Sensing Object Detection via Selective-Perspective-Class Integrationcited by 0
  9. 2025ACM-UNet: Adaptive Integration of CNNs and Mamba for Efficient Medical Image Segmentationcited by 0
  10. 2025O2Former:Direction-Aware and Multi-Scale Query Enhancement for SAR Ship Instance Segmentationcited by 0
  11. 2025DGE-YOLO: Dual-Branch Gathering and Attention for Accurate UAV Object Detectioncited by 0
  12. 2025DiffFit: Disentangled Garment Warping and Texture Refinement for Virtual Try-Oncited by 0
  13. 2025OptiPrune: Boosting Prompt-Image Consistency with Attention-Guided Noise and Dynamic Token Selectioncited by 0
  14. 2025FashionPose: Unified Text-Driven Fashion Synthesis with Joint Geometric and Photometric Controlcited by 0
  15. 2025Conditional Diffusion Model with Anatomical-Dose Dual Constraints for End-to-End Multi-Tumor Dose Predictioncited by 0
  16. 2025Animalbooth: multimodal feature enhancement for animal subject personalizationcited by 0
  17. 2025DENet: Dual-Path Edge Network with Global-Local Attention for Infrared Small Target Detectioncited by 0
  18. 2025Instruction Guided Multi Object Image Editing with Quantity and Layout Consistencycited by 0
  19. 2025Span-level Detection of AI-generated Scientific Text via Contrastive Learning and Structural Calibrationcited by 0
  20. 2025SynergyWarpNet: Attention-Guided Cooperative Warping for Neural Portrait Animationcited by 0
  21. 2025ManchuTTS: Towards High-Quality Manchu Speech Synthesis via Flow Matching and Hierarchical Text Representationcited by 0
  22. 2025GCA-ResUNet: Medical Image Segmentation Using Grouped Coordinate Attentioncited by 0
  23. 2026Causal Discovery for Cross-Sectional Data Based on Super-Structure and Divide-and-Conquercited by 0
  24. 2026FGAA-FPN: Foreground-Guided Angle-Aware Feature Pyramid Network for Oriented Object Detectioncited by 0
  25. 2026Multimodal Consistency-Guided Reference-Free Data Selection for ASR Accent Adaptationcited by 0
  26. 2026FinSight-Net:A Physics-Aware Decoupled Network with Frequency-Domain Compensation for Underwater Fish Detection in Smart Aquaculturecited by 0
  27. 2026RSTNet: Enhancing Small-Target Recognition in Noisy SAR Imagery via Robust Feature Learning and Distribution-Aware Regressioncited by 0
  28. 2026LiftAvatar: Kinematic-Space Completion for Expression-Controlled 3D Gaussian Avatar Animationcited by 0
  29. 2026Multi-Period Texture Contrast Enhancement for Low-Contrast Wafer Defect Detection and Segmentationcited by 0
  30. 2026SARES-DEIM: Sparse Mixture-of-Experts Meets DETR for Robust SAR Ship Detectioncited by 0
  31. 2026Beyond Reconstruction: Reconstruction-to-Vector Diffusion for Hyperspectral Anomaly Detectioncited by 0
  32. 2026WILD-SAM: Phase-Aware Expert Adaptation of SAM for Landslide Detection in Wrapped InSAR Interferogramscited by 0
  33. 2026Efficient Task Adaptation in Large Language Models via Selective Parameter Optimizationcited by 0
  34. 2026LER-YOLO: Reliability-Aware Expert Routing for Misaligned RGB-Infrared UAV Detectioncited by 0
  35. 2026IB-HFN: Information Bottleneck-Driven SAR-Optical Fusion Network for High-Fidelity Cloud Removalcited by 0
  36. 2026TRNet: Learning with Topographic Priors for VHR Paddy Rice Mappingcited by 0
  37. 2026Dual Modality Prompted Diffusion Priors for Zero Shot Hyperspectral Pansharpeningcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2022Talking Head Generation with Probabilistic Audio-to-Visual Diffusion Priors
  2. 2024IF-MDM: Implicit Face Motion Diffusion Model for High-Fidelity Realtime Talking Head Generation
  3. 2024MEMO: Memory-Guided Diffusion for Expressive Talking Video Generation
  4. 2025AI killed the video star. Audio-driven diffusion model for expressive talking head generation
  5. 2024DAWN: Dynamic Frame Avatar with Non-autoregressive Diffusion Framework for Talking Head Video Generation
  6. 2025See the Speaker: Crafting High-Resolution Talking Faces from Speech with Prior Guidance and Region Refinement
  7. 2025Dimitra: Audio-driven Diffusion model for Expressive Talking Head Generation
  8. 2026TempoSyncDiff: Distilled Temporally-Consistent Diffusion for Low-Latency Audio-Driven Talking Head Generation