X-Actor: Emotional and Expressive Long-Range Portrait Acting from Audio
On the map: Diffusion models → Audio, speech and music
Abstract (arXiv)
We present X-Actor, a novel audio-driven portrait animation framework that generates lifelike, emotionally expressive talking head videos from a single reference image and an input audio clip. Unlike prior methods that emphasize lip synchronization and short-range visual fidelity in constrained speaking scenarios, X-Actor enables actor-quality, long-form portrait performance capturing nuanced, dynamically evolving emotions that flow coherently with the rhythm and content of speech. Central to our approach is a two-stage decoupled generation pipeline: an audio-conditioned autoregressive diffusion model that predicts expressive yet identity-agnostic facial motion latent tokens within a long temporal context window, followed by a diffusion-based video synthesis module that translates these motions into high-fidelity video animations. By operating in a compact facial motion latent space decoupled from visual and identity cues, our autoregressive diffusion model effectively captures long-range correlations between audio and facial dynamics through a diffusion-forcing training paradigm, enabling infinite-length emotionally-rich motion prediction without error accumulation. Extensive experiments demonstrate that X-Actor produces compelling, cinematic-style performances that go beyond standard talking head animations and achieves state-of-the-art results in long-range, audio-driven emotional portrait acting.
Key facts. Cited by 3 papers in the map (data through 14 Oct 2026). Placed in Audio, speech and music, one of 14 directions in Diffusion models. Builds on 32 of its 51 references that are in the map. First posted on arXiv 4 Aug 2025.
Builds on
32 of 51 references.
Closest ancestors · 4
Same direction, newest first
- 2024MEMO: Memory-Guided Diffusion for Expressive Talking Video Generation5 Dec
- 2024Hallo3: Highly Dynamic and Realistic Portrait Image Animation with Video Diffusion Transformer1 Dec
- 2024Hallo2: Long-Duration and High-Resolution Audio-Driven Portrait Image Animation10 Oct
- 2024Loopy: Taming Audio-Driven Portrait Avatar with Long-Term Motion Dependency4 Sep
Other references · 28
- 2020Denoising Diffusion Implicit Modelscited by 9,781
- 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,905
- 2023AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuningcited by 1,433
- 2023MasaCtrl: Tuning-Free Mutual Self-Attention Control for Consistent Image Synthesis and Editingcited by 727
- 2023Animate Anyone: Consistent and Controllable Image-to-Video Synthesis for Character Animationcited by 688
- 2024Diffusion Forcing: Next-token Prediction Meets Full-Sequence Diffusioncited by 681
- 2020A Lip Sync Expert Is All You Need for Speech to Lip Generation In The Wildcited by 672
- 2020One-Shot Free-View Neural Talking-Head Synthesis for Video Conferencingcited by 444
- 2022SadTalker: Learning Realistic 3D Motion Coefficients for Stylized Audio-Driven Single Image Talking Face Animationcited by 382
- 2020MakeItTalk: Speaker-Aware Talking-Head Animationcited by 312
- 2022CelebV-HQ: A Large-Scale Video Facial Attributes Datasetcited by 250
- 2024EchoMimic: Lifelike Audio-Driven Portrait Animations through Editable Landmark Conditionscited by 196
16 more
- 2024AniPortrait: Audio-Driven Synthesis of Photorealistic Portrait Animationcited by 194
- 2024VASA-1: Lifelike Audio-Driven Talking Faces Generated in Real Timecited by 187
- 2022StyleHEAT: One-Shot High-Resolution Editable Talking Face Generation via Pre-trained StyleGANcited by 169
- 2018Lip Movements Generation at a Glancecited by 155
- 2022MegaPortraits: One-shot Megapixel Neural Head Avatarscited by 131
- 2022VideoReTalking: Audio-based Lip Synchronization for Talking Head Video Editing In the Wildcited by 106
- 2021FACIAL: Synthesizing Dynamic Talking Face with Implicit Attribute Learningcited by 90
- 2024V-Express: Conditional Dropout for Progressive Training of Portrait Video Generationcited by 86
- 2025FantasyTalking: Realistic Talking Portrait Generation via Coherent Motion Synthesiscited by 77
- 2023VividTalk: One-Shot Audio-Driven Talking Head Generation Based on 3D Hybrid Priorcited by 47
- 2023GAIA: Zero-shot Talking Avatar Generationcited by 43
- 2025EMO2: End-Effector Guided Audio-Driven Avatar Video Generationcited by 33
- 2025X-NeMo: Expressive Neural Motion Reenactment via Disentangled Latent Attentioncited by 28
- 2024AsyncDiff: Parallelizing Diffusion Models by Asynchronous Denoisingcited by 22
- 2025Teller: Real-Time Streaming Audio-Driven Portrait Animation with Autoregressive Motion Generationcited by 16
- 2024JoyVASA: Portrait and Animal Image Animation with Diffusion-Based Audio-Driven Facial Dynamics and Head Motion Generationcited by 9
Built on by
Citing papers per year: 2025: 1; 2026: 2.
3 papers.
- 2025X-Streamer: Unified Human World Modeling with Audiovisual Interactioncited by 18
- 2026AvatarForcing: One-Step Streaming Talking Avatars via Local-Future Sliding-Window Denoisingcited by 6
- 2026HairWeaver: Few-Shot Photorealistic Hair Motion Synthesis with Sim-to-Real Guided Video Diffusioncited by 1
- 2026AvatarForcing: One-Step Streaming Talking Avatars via Local-Future Sliding-Window Denoising15 Mar 2026
- 2026HairWeaver: Few-Shot Photorealistic Hair Motion Synthesis with Sim-to-Real Guided Video Diffusion11 Feb 2026
- 2025X-Streamer: Unified Human World Modeling with Audiovisual Interaction25 Sep 2025
Similar, not cited
Close in wording, no citation either way
- 2024EMOdiffhead: Continuously Emotional Control in Talking Head Generation via Diffusion
- 2024X-Portrait: Expressive Portrait Animation with Hierarchical Motion Attention
- 2024GoHD: Gaze-oriented and Highly Disentangled Portrait Animation with Rhythmic Poses and Realistic Expression
- 2024Hallo: Hierarchical Audio-Driven Visual Synthesis for Portrait Image Animation
- 2024EMO: Emote Portrait Alive -- Generating Expressive Portrait Videos with Audio2Video Diffusion Model under Weak Conditions
- 2024EmotiveTalk: Expressive Talking Head Generation through Audio Information Decoupling and Emotional Video Diffusion
- 2025AI killed the video star. Audio-driven diffusion model for expressive talking head generation
- 2024DAWN: Dynamic Frame Avatar with Non-autoregressive Diffusion Framework for Talking Head Video Generation