ECHO: Towards Emotionally Appropriate and Contextually Aware Interactive Head Generation
Not on the map. Why?
Abstract (arXiv)
In natural face-to-face interaction, participants seamlessly alternate between speaking and listening, producing facial behaviors (FBs) that are finely informed by long-range context and naturally exhibit contextual appropriateness and emotional rationality. Interactive Head Generation (IHG) aims to synthesize lifelike avatar head video emulating such capabilities. Existing IHG methods typically condition on dual-track signals (i.e., human user's behaviors and pre-defined audio for avatar) within a short temporal window, jointly driving generation of avatar's audio-aligned lip articulation and non-verbal FBs. However, two main challenges persist in these methods: (i) the reliance on short-clip behavioral cues without long-range contextual modeling leads them to produce facial behaviors lacking contextual appropriateness; and (ii) the entangled, role-agnostic fusion of dual-track signals empirically introduces cross-signal interference, potentially compromising lip-region synchronization during speaking. To this end, we propose ECHO, a novel IHG framework comprising two key components: a Long-range Contextual Understanding (LCU) component that facilitates contextual understanding of both behavior-grounded dynamics and linguistic-driven affective semantics to promote contextual appropriateness and emotional rationality of synthesized avatar FBs; and a block-wise Spatial-aware Decoupled Cross-attention Modulation (SDCM) module, that preserves self-audio-driven lip articulation while adaptively integrating user contextual behavioral cues for non-lip facial regions, complemented by our designed two-stage training paradigm, to jointly enhance lip synchronization and visual fidelity. Extensive experiments demonstrate the effectiveness of proposed components and ECHO's superior IHG performance.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 42 of its 56 references that are in the map. First posted on arXiv 18 Mar 2026.
Builds on
42 of 56 references.
References · 42
- 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
- 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
- 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
- 2022Scalable Diffusion Models with Transformerscited by 5,931
- 2023Mamba: Linear-Time Sequence Modeling with Selective State Spacescited by 4,616
- 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,853
- 2018Towards Accurate Generative Models of Video: A New Metric & Challengescited by 1,204
- 2023One-step Diffusion with Distribution Matching Distillationcited by 978
- 2025Qwen2.5-Omni Technical Reportcited by 704
- 2024Improved Distribution Matching Distillation for Fast Image Synthesiscited by 648
- 2019MediaPipe: A Framework for Building Perception Pipelinescited by 572
- 2017How far are we from solving the 2D & 3D Face Alignment problem? (and a dataset of 230,000 3D facial landmarks)cited by 524
30 more
- 2019Accurate 3D Face Reconstruction with Weakly-Supervised Learning: From Single Image to Image Setcited by 481
- 2025Qwen3-Omni Technical Reportcited by 451
- 2024From Slow Bidirectional to Fast Autoregressive Video Diffusion Modelscited by 401
- 2022SadTalker: Learning Realistic 3D Motion Coefficients for Stylized Audio-Driven Single Image Talking Face Animationcited by 380
- 2020MakeItTalk: Speaker-Aware Talking-Head Animationcited by 310
- 2022EMOCA: Emotion Driven Monocular Face Capture and Animationcited by 254
- 2017Gated Multimodal Units for Information Fusioncited by 244
- 2024LivePortrait: Efficient Portrait Animation with Stitching and Retargeting Controlcited by 202
- 2024Hallo: Hierarchical Audio-Driven Visual Synthesis for Portrait Image Animationcited by 198
- 2025OmniHuman-1: Rethinking the Scaling-Up of One-Stage Conditioned Human Animation Modelscited by 127
- 2022Learning to Listen: Modeling Non-Deterministic Dyadic Facial Motioncited by 125
- 2024Hallo3: Highly Dynamic and Realistic Portrait Image Animation with Video Diffusion Transformercited by 121
- 2021Responsive Listening Head Generation: A Benchmark Dataset and Baselinecited by 67
- 2023Can Language Models Learn to Listen?cited by 40
- 2024FLOAT: Generative Motion Latent Flow Matching for Audio-driven Talking Portraitcited by 40
- 2024INFP: Audio-Driven Interactive Head Generation in Dyadic Conversationscited by 40
- 2025Seamless Interaction: Dyadic Audiovisual Motion Modeling and Large-Scale Datasetcited by 35
- 2023Affective Faces for Goal-Driven Dyadic Communicationcited by 34
- 2024Dyadic Interaction Modeling for Social Behavior Generationcited by 32
- 2025DualTalk: Dual-Speaker Interaction for 3D Talking Head Conversationscited by 31
- 2025OmniHuman-1.5: Instilling an Active Mind in Avatars via Cognitive Simulationcited by 30
- 2025StreamAvatar: Streaming Diffusion Models for Real-Time Interactive Human Avatarscited by 24
- 2024CustomListener: Text-guided Responsive Interaction for User-friendly Listening Head Generationcited by 22
- 2023ReactFace: Online Multiple Appropriate Facial Reaction Generation in Dyadic Interactionscited by 17
- 2026Avatar Forcing: Real-Time Interactive Head Avatar Generation for Natural Conversationcited by 17
- 2025ARIG: Autoregressive Interactive Head Generation for Real-time Conversationscited by 16
- 2023Interactive Conversational Head Generationcited by 14
- 2025REACT 2025: the Third Multiple Appropriate Facial Reaction Generation Challengecited by 9
- 2025Towards Interactive Intelligence for Digital Humanscited by 8
- 2025LiveTalk: Real-Time Multimodal Interactive Video Diffusion via Improved On-Policy Distillationcited by 8
Built on by
Not cited yet.