InfinityHuman: Towards Long-Term Audio-Driven Human
Not on the map. Why?
Abstract (arXiv)
Audio-driven human animation has attracted wide attention thanks to its practical applications. However, critical challenges remain in generating high-resolution, long-duration videos with consistent appearance and natural hand motions. Existing methods extend videos using overlapping motion frames but suffer from error accumulation, leading to identity drift, color shifts, and scene instability. Additionally, hand movements are poorly modeled, resulting in noticeable distortions and misalignment with the audio. In this work, we propose InfinityHuman, a coarse-to-fine framework that first generates audio-synchronized representations, then progressively refines them into high-resolution, long-duration videos using a pose-guided refiner. Since pose sequences are decoupled from appearance and resist temporal degradation, our pose-guided refiner employs stable poses and the initial frame as a visual anchor to reduce drift and improve lip synchronization. Moreover, to enhance semantic accuracy and gesture realism, we introduce a hand-specific reward mechanism trained with high-quality hand motion data. Experiments on the EMTD and HDTF datasets show that InfinityHuman achieves state-of-the-art performance in video quality, identity preservation, hand accuracy, and lip-sync. Ablation studies further confirm the effectiveness of each module. Code will be made public.
Key facts. Cited by 9 papers in the map (data through 14 Oct 2026). Builds on 31 of its 40 references that are in the map. First posted on arXiv 27 Aug 2025.
Builds on
31 of 40 references.
References · 31
- 2018Towards Accurate Generative Models of Video: A New Metric & Challengescited by 1,234
- 2023Animate Anyone: Consistent and Controllable Image-to-Video Synthesis for Character Animationcited by 688
- 2023Q-Align: Teaching LMMs for Visual Scoring via Discrete Text-Defined Levelscited by 503
- 2024Panda-70M: Captioning 70M Videos with Multiple Cross-Modality Teacherscited by 393
- 2022SadTalker: Learning Realistic 3D Motion Coefficients for Stylized Audio-Driven Single Image Talking Face Animationcited by 382
- 2024Fast High-Resolution Image Synthesis with Latent Adversarial Diffusion Distillationcited by 279
- 2024StreamingT2V: Consistent, Dynamic, and Extendable Long Video Generation from Textcited by 251
- 2024Sapiens: Foundation for Human Vision Modelscited by 229
- 2024Hallo: Hierarchical Audio-Driven Visual Synthesis for Portrait Image Animationcited by 202
- 2024EchoMimic: Lifelike Audio-Driven Portrait Animations through Editable Landmark Conditionscited by 196
- 2024AniPortrait: Audio-Driven Synthesis of Photorealistic Portrait Animationcited by 194
- 2023FreeNoise: Tuning-Free Longer Video Diffusion via Noise Reschedulingcited by 180
19 more
- 2024Vidu: a Highly Consistent, Dynamic and Skilled Text-to-Video Generator with Diffusion Modelscited by 160
- 2024Hyper-SD: Trajectory Segmented Consistency Model for Efficient Image Synthesiscited by 156
- 2025Frame Context Packing and Drift Prevention in Next-Frame-Prediction Video Diffusion Modelscited by 143
- 2023Make Pixels Dance: High-Dynamic Video Generationcited by 140
- 2023Gen-L-Video: Multi-Text to Long Video Generation via Temporal Co-Denoisingcited by 139
- 2025OmniHuman-1: Rethinking the Scaling-Up of One-Stage Conditioned Human Animation Modelscited by 129
- 2024Hallo3: Highly Dynamic and Realistic Portrait Image Animation with Video Diffusion Transformercited by 122
- 2024Loopy: Taming Audio-Driven Portrait Avatar with Long-Term Motion Dependencycited by 102
- 2024V-Express: Conditional Dropout for Progressive Training of Portrait Video Generationcited by 86
- 2025OmniAvatar: Efficient Audio-Driven Avatar Video Generation with Adaptive Body Animationcited by 83
- 2025FantasyTalking: Realistic Talking Portrait Generation via Coherent Motion Synthesiscited by 77
- 2024EchoMimicV2: Towards Striking, Simplified, and Semi-Body Human Animationcited by 72
- 2025HunyuanVideo-Avatar: High-Fidelity Audio-Driven Human Animation for Multiple Characterscited by 69
- 2025Goku: Flow Based Video Generative Foundation Modelscited by 67
- 2025Let Them Talk: Audio-Driven Multi-Person Conversational Video Generationcited by 66
- 2024ConsistentID: Portrait Generation with Multimodal Fine-Grained Identity Preservingcited by 43
- 2024CyberHost: Taming Audio-driven Avatar Diffusion Model with Region Codebook Attentioncited by 39
- 2025HumanDiT: Pose-Guided Diffusion Transformer for Long-form Human Motion Video Generationcited by 39
- 2024DiffTED: One-shot Audio-driven TED Talk Video Generation with Diffusion-based Co-speech Gesturescited by 20
Built on by
Citing papers per year: 2025: 4; 2026: 5.
9 papers.
- 2026Flow caching for autoregressive video generationcited by 25
- 2025SpatialTree: How Spatial Abilities Branch Out in MLLMscited by 12
- 2025ViBES: A Conversational Agent with Behaviorally-Intelligent 3D Virtual Bodycited by 6
- 2025Soul: Breathe Life into Digital Human for High-fidelity Long-term Multimodal Animationcited by 3
- 2025SyncAnyone: Implicit Disentanglement via Progressive Self-Correction for Lip-Syncing in the wildcited by 2
- 2026InterDyad: Interactive Dyadic Speech-to-Video Generation by Querying Intermediate Visual Guidancecited by 1
- 2026EchoTorrent: Towards Swift, Sustained, and Streaming Multi-Modal Video Generationcited by 0
- 2026Generate Your Talking Avatar from Video Referencecited by 0
- 2026SyncCache: Exploiting Asymmetric Dynamics for Fast Audio-Driven Portrait Animationcited by 0
- 2026SyncCache: Exploiting Asymmetric Dynamics for Fast Audio-Driven Portrait Animation29 Jun 2026
- 2026Generate Your Talking Avatar from Video Reference30 Apr 2026
- 2026InterDyad: Interactive Dyadic Speech-to-Video Generation by Querying Intermediate Visual Guidance24 Mar 2026
- 2026EchoTorrent: Towards Swift, Sustained, and Streaming Multi-Modal Video Generation14 Feb 2026
- 2026Flow caching for autoregressive video generation11 Feb 2026
- 2025SyncAnyone: Implicit Disentanglement via Progressive Self-Correction for Lip-Syncing in the wild25 Dec 2025
- 2025SpatialTree: How Spatial Abilities Branch Out in MLLMs23 Dec 2025
- 2025ViBES: A Conversational Agent with Behaviorally-Intelligent 3D Virtual Body16 Dec 2025
- 2025Soul: Breathe Life into Digital Human for High-fidelity Long-term Multimodal Animation15 Dec 2025