mlsanityData to 7 Oct 2026

Wan-S2V: Audio-Driven Cinematic Video Generation

arXiv 2508.18621 ↗First posted 26 Aug 2025cs.CVcited by 60

Not on the map. Why?

Abstract (arXiv)

Current state-of-the-art (SOTA) methods for audio-driven character animation demonstrate promising performance for scenarios primarily involving speech and singing. However, they often fall short in more complex film and television productions, which demand sophisticated elements such as nuanced character interactions, realistic body movements, and dynamic camera work. To address this long-standing challenge of achieving film-level character animation, we propose an audio-driven model, which we refere to as Wan-S2V, built upon Wan. Our model achieves significantly enhanced expressiveness and fidelity in cinematic contexts compared to existing approaches. We conducted extensive experiments, benchmarking our method against cutting-edge models such as Hunyuan-Avatar and Omnihuman. The experimental results consistently demonstrate that our approach significantly outperforms these existing solutions. Additionally, we explore the versatility of our method through its applications in long-form video generation and precise video lip-sync editing.

Key facts. Cited by 60 papers in the map (data through 7 Oct 2026). Builds on 16 of its 25 references that are in the map. First posted on arXiv 26 Aug 2025.

Builds on

16 of 25 references.

References · 16

  1. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  2. 2025Qwen2.5-VL Technical Reportcited by 5,418
  3. 2025Wan: Open and Advanced Large-Scale Video Generative Modelscited by 2,536
  4. 2022ViTPose: Simple Vision Transformer Baselines for Human Pose Estimationcited by 450
  5. 2022Exploring Video Quality Assessment on User Generated Contents from Aesthetic and Technical Perspectivescited by 314
  6. 2023Effective Whole-body Pose Estimation with Two-stages Distillationcited by 307
  7. 2022Unifying Flow, Stereo and Depth Estimationcited by 270
  8. 2024EMO: Emote Portrait Alive -- Generating Expressive Portrait Videos with Audio2Video Diffusion Model under Weak Conditionscited by 233
  9. 2024MimicMotion: High-Quality Human Motion Video Generation with Confidence-aware Pose Guidancecited by 196
  10. 2024Koala-36M: A Large-scale Video Dataset Improving Consistency between Fine-grained Conditions and Video Contentcited by 132
  11. 2025OmniHuman-1: Rethinking the Scaling-Up of One-Stage Conditioned Human Animation Modelscited by 127
  12. 2025Frame Context Packing and Drift Prevention in Next-Frame-Prediction Video Diffusion Modelscited by 126
4 more
  1. 2025FantasyTalking: Realistic Talking Portrait Generation via Coherent Motion Synthesiscited by 77
  2. 2025HunyuanVideo-Avatar: High-Fidelity Audio-Driven Human Animation for Multiple Characterscited by 64
  3. 2023A Light Weight Model for Active Speaker Detectioncited by 39
  4. 2025EMO2: End-Effector Guided Audio-Driven Avatar Video Generationcited by 32

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 19; 2026: 41.

60 papers.

Sort:

By area:

  1. 2026LTX-2: Efficient Joint Audio-Visual Foundation Modelcited by 213
  2. 2025UniVerse-1: Unified Audio-Video Generation via Stitching of Expertscited by 58
  3. 2025UniAVGen: Unified Audio and Video Generation with Asymmetric Cross-Modal Interactionscited by 35
  4. 2026JavisDiT++: Unified Modeling and Optimization for Joint Audio-Video Generationcited by 22
  5. 2025Kling-Avatar: Grounding Multimodal Instructions for Cascaded Long-Duration Avatar Animation Synthesiscited by 20
  6. 2025AnyTalker: Scaling Multi-Person Talking Video Generation with Interactivity Refinementcited by 19
  7. 2026Mode Seeking meets Mean Seeking for Fast Long Video Generationcited by 12
  8. 2025KlingAvatar 2.0 Technical Reportcited by 11
  9. 2025Knot Forcing: Taming Autoregressive Video Diffusion Models for Real-time Infinite Interactive Portrait Animationcited by 11
  10. 2026LPM 1.0: Video-based Character Performance Modelcited by 11
  11. 2025SoulX-FlashTalk: Real-Time Infinite Streaming of Audio-Driven Avatars via Self-Correcting Bidirectional Distillationcited by 10
  12. 2025LinVideo: A Post-Training Framework towards O(n) Attention in Efficient Video Generationcited by 9
38 more
  1. 2025ASTRA: Let Arbitrary Subjects Transform in Video Editingcited by 6
  2. 2026AvatarForcing: One-Step Streaming Talking Avatars via Local-Future Sliding-Window Denoisingcited by 6
  3. 2025ActAvatar: Temporally-Aware Precise Action Control for Talking Avatarscited by 5
  4. 2026LongCat-Video-Avatar 1.5 Technical Reportcited by 5
  5. 20253MDiT: Unified Tri-Modal Diffusion Transformer for Text-Driven Synchronized Audio-Video Generationcited by 4
  6. 2025MAViD: A Multimodal Framework for Audio-Visual Dialogue Understanding and Generationcited by 4
  7. 2026SoulX-FlashHead: Oracle-guided Generation of Infinite Real-time Streaming Talking Headscited by 4
  8. 2026Improving Joint Audio-Video Generation with Cross-Modal Context Learningcited by 4
  9. 2026Vidu S1: A Real-Time Interactive Video Generation Modelcited by 4
  10. 2025Understanding, Accelerating, and Improving MeanFlow Trainingcited by 3
  11. 2025JoyStreamer-Flash: Real-time and Infinite Audio-Driven Avatar Generation with Autoregressive Diffusioncited by 3
  12. 2025Soul: Breathe Life into Digital Human for High-fidelity Long-term Multimodal Animationcited by 3
  13. 2026Making Avatars Interact: Towards Text-Driven Human-Object Interaction for Controllable Talking Avatarscited by 3
  14. 2026EA-WM: Event-Aware Generative World Model with Structured Kinematic-to-Visual Action Fieldscited by 3
  15. 2026StreamChar: Long-Horizon Streaming Character Audio-Video Generation with Decoupled Orchestrationcited by 3
  16. 2026OmniDance: Multimodal Driven Dance Video Generation with Large-scale Internet Datacited by 3
  17. 2025Live Avatar: Streaming Real-time Audio-Driven Avatar Generation with Infinite Lengthcited by 2
  18. 2026CoInteract: Physically-Consistent Human-Object Interaction Video Synthesis via Spatially-Structured Co-Generationcited by 2
  19. 2026Omni-Customizer: End-to-End MultiModal Customization for Joint Audio-Video Generationcited by 2
  20. 2025TalkVerse: Democratizing Minute-Long Audio-Driven Video Generationcited by 1
  21. 2025SyncAnyone: Implicit Disentanglement via Progressive Self-Correction for Lip-Syncing in the wildcited by 1
  22. 2026AUHead: Realistic Emotional Talking Head Generation via Action Units Controlcited by 1
  23. 2026FlexAM: Flexible Appearance-Motion Decomposition for Versatile Video Generation Controlcited by 1
  24. 2026InterDyad: Interactive Dyadic Speech-to-Video Generation by Querying Intermediate Visual Guidancecited by 1
  25. 2026Mutual Forcing: Dual-Mode Self-Evolution for Fast Autoregressive Audio-Video Character Generationcited by 1
  26. 2026AdvDMD: Adversarial Reward Meets DMD For High-Quality Few-Step Generationcited by 1
  27. 2026VidSplat: Gaussian Splatting Reconstruction with Geometry-Guided Video Diffusion Priorscited by 1
  28. 2026Ink3D: Sculpting 3D Assets with Extremely Complex Textures via Video Generative Modelscited by 1
  29. 2026Wan-Dancer: A Hierarchical Framework for Minute-scale Coherent Music-to-Dance Generationcited by 1
  30. 2026AptAvatar: Fast and Vivid Long-Form Audio-Driven Video Generation for Production-Ready Avatarscited by 1
  31. 2026DynaForcing: Overcoming Dynamic Collapse in Self-Forcing Distillation for Streaming Avatar Generationcited by 1
  32. 2025JoVA: Unified Multimodal Learning for Joint Video-Audio Generation and Editingcited by 0
  33. 2026JoyStreamer: Unlocking Highly Expressive Avatars via Harmonized Text-Audio Conditioningcited by 0
  34. 2026Asymmetric Hierarchical Anchoring for Robust Audio-Visual Cross-Modal Generalizationcited by 0
  35. 2026EchoTorrent: Towards Swift, Sustained, and Streaming Multi-Modal Video Generationcited by 0
  36. 2026Tora3: Trajectory-Guided Audio-Video Generation with Physical Coherencecited by 0
  37. 2026Generate Your Talking Avatar from Video Referencecited by 0
  38. 2026SyncCache: Exploiting Asymmetric Dynamics for Fast Audio-Driven Portrait Animationcited by 0

50 most cited of 60 All 60 (JSON)