Avatar V: Scaling Video-Reference Avatar Video Generation
Not on the map. Why?
Abstract (arXiv)
Generating avatar videos that are not merely visually similar to a target individual but behaviorally recognizable, faithfully reproducing their talking rhythm, gestural tendencies, and expression dynamics, remains an open challenge. Existing methods predominantly condition on single static images, which provide insufficient identity information and cannot capture dynamic motion traits, while standard pixel-level objectives underserve the perceptually critical facial regions that determine avatar fidelity. We present Avatar V, a production-scale framework that addresses these limitations through video-reference-conditioned identity modeling. Rather than compressing identity into fixed-size embeddings, the model conditions directly on the full token sequence of a reference video, learning to reproduce both static identity attributes (facial geometry, skin texture) and dynamic behavioral patterns (talking rhythm, micro-expressions) through attention over the reference context. We introduce Sparse Reference Attention, an asymmetric mechanism achieving linear-complexity conditioning on arbitrarily long references; a motion representation stream enabling closed-loop talking style transfer; and an identity-aware super-resolution refiner inheriting the full reference conditioning. These are supported by a data engine curating 100M+ training clips from 50M raw videos, and a five-stage training pipeline with flow matching pre-training, personality fine-tuning, two-phase distillation (>10x acceleration), and RLHF alignment, deployed across thousands of GPUs. Avatar V generates 1080p videos of unlimited duration, achieving state-of-the-art identity preservation, lip synchronization, and generation quality on our cross-scene benchmark, consistently outperforming leading systems including Seedance 2.0, Kling O3 Pro, Veo 3.1, and OmniHuman 1.5 in both automated metrics and human evaluation.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 61 of its 71 references that are in the map. First posted on arXiv 11 Jun 2026.
Builds on
61 of 71 references.
References · 61
- 2020Denoising Diffusion Probabilistic Modelscited by 20,851
- 2024DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Modelscited by 8,851
- 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 7,916
- 2022Scalable Diffusion Models with Transformerscited by 5,931
- 2022Classifier-Free Diffusion Guidancecited by 5,717
- 2022Flow Matching for Generative Modelingcited by 5,426
- 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,094
- 2025Wan: Open and Advanced Large-Scale Video Generative Modelscited by 2,536
- 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasetscited by 2,526
- 2024CogVideoX: Text-to-Video Diffusion Models with An Expert Transformercited by 2,198
- 2023Consistency Modelscited by 1,778
- 2024HunyuanVideo: A Systematic Framework For Large Video Generative Modelscited by 1,597
38 more
- 2023IP-Adapter: Text Compatible Image Prompt Adapter for Text-to-Image Diffusion Modelscited by 1,439
- 2023AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuningcited by 1,418
- 2023One-step Diffusion with Distribution Matching Distillationcited by 978
- 2023Training Diffusion Models with Reinforcement Learningcited by 939
- 2025Cosmos World Foundation Model Platform for Physical AIcited by 833
- 2023Diffusion Model Alignment Using Direct Preference Optimizationcited by 767
- 2022On Distillation of Guided Diffusion Modelscited by 721
- 2023Animate Anyone: Consistent and Controllable Image-to-Video Synthesis for Character Animationcited by 683
- 2024Open-Sora: Democratizing Efficient Video Production for Allcited by 637
- 2024Movie Gen: A Cast of Media Foundation Modelscited by 614
- 2025Flow-GRPO: Training Flow Matching Models via Online RLcited by 605
- 2024Latte: Latent Diffusion Transformer for Video Generationcited by 536
- 2023Q-Align: Teaching LMMs for Visual Scoring via Discrete Text-Defined Levelscited by 489
- 2023DynamiCrafter: Animating Open-domain Images with Video Diffusion Priorscited by 463
- 2024Lumiere: A Space-Time Diffusion Model for Video Generationcited by 450
- 2024InstantID: Zero-shot Identity-Preserving Generation in Secondscited by 431
- 2025DanceGRPO: Unleashing GRPO on Visual Generationcited by 388
- 2022SadTalker: Learning Realistic 3D Motion Coefficients for Stylized Audio-Driven Single Image Talking Face Animationcited by 380
- 2024Pyramidal Flow Matching for Efficient Video Generative Modelingcited by 354
- 2023MagicAnimate: Temporally Consistent Human Image Animation using Diffusion Modelcited by 330
- 2023PhotoMaker: Customizing Realistic Human Photos via Stacked ID Embeddingcited by 324
- 2024Champ: Controllable and Consistent Human Image Animation with 3D Parametric Guidancecited by 271
- 2025Seedance 1.0: Exploring the Boundaries of Video Generation Modelscited by 247
- 2024EMO: Emote Portrait Alive -- Generating Expressive Portrait Videos with Audio2Video Diffusion Model under Weak Conditionscited by 233
- 2025SkyReels-V2: Infinite-length Film Generative Modelcited by 218
- 2024LivePortrait: Efficient Portrait Animation with Stitching and Retargeting Controlcited by 202
- 2024Hallo: Hierarchical Audio-Driven Visual Synthesis for Portrait Image Animationcited by 198
- 2024MimicMotion: High-Quality Human Motion Video Generation with Confidence-aware Pose Guidancecited by 196
- 2024EchoMimic: Lifelike Audio-Driven Portrait Animations through Editable Landmark Conditionscited by 196
- 2024AniPortrait: Audio-Driven Synthesis of Photorealistic Portrait Animationcited by 191
- 2024VASA-1: Lifelike Audio-Driven Talking Faces Generated in Real Timecited by 186
- 2025Open-Sora 2.0: Training a Commercial-Level Video Generation Model in 200kcited by 157
- 2025OmniHuman-1: Rethinking the Scaling-Up of One-Stage Conditioned Human Animation Modelscited by 127
- 2024Hallo3: Highly Dynamic and Realistic Portrait Image Animation with Video Diffusion Transformercited by 121
- 2024UniAnimate: Taming Unified Video Diffusion Models for Consistent Human Image Animationcited by 101
- 2024Hallo2: Long-Duration and High-Resolution Audio-Driven Portrait Image Animationcited by 95
- 2024X-Portrait: Expressive Portrait Animation with Hierarchical Motion Attentioncited by 88
- 2024V-Express: Conditional Dropout for Progressive Training of Portrait Video Generationcited by 85
50 most cited of 61 All 61 (JSON)
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2024DAWN: Dynamic Frame Avatar with Non-autoregressive Diffusion Framework for Talking Head Video Generation
- 2024One Shot, One Talk: Whole-body Talking Avatar from a Single Image
- 2024Make-Your-Anchor: A Diffusion-based 2D Avatar Generation Framework
- 2026Avatar Forcing: Real-Time Interactive Head Avatar Generation for Natural Conversation
- 2023Dancing Avatar: Pose and Text-Guided Human Motion Videos Synthesis with Image Diffusion Model
- 2024VLOGGER: Multimodal Diffusion for Embodied Avatar Synthesis
- 2026Where and When to Force: Routed Forcing for Streaming Avatars
- 2026LiftAvatar: Kinematic-Space Completion for Expression-Controlled 3D Gaussian Avatar Animation