# ECHO: Dyadic 3D Facial Motion Generation with Asymmetric Deterministic Articulation and Stochastic Reaction

arXiv: https://arxiv.org/abs/2609.05506 · HTML: https://mlsanity.com/paper/2609.05506/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 26 of its 49 references that are in the map.
- First posted on arXiv 29 Aug 2026.

## Builds on (most cited first, 26 of 26)

- [Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flow](https://mlsanity.com/paper/2209.03003/) (2022)
- [HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units](https://mlsanity.com/paper/2106.07447/) (2021)
- [Capture, Learning, and Synthesis of 3D Speaking Styles](https://mlsanity.com/paper/1905.03079/) (2019)
- [MeshTalk: 3D Face Animation from Speech using Cross-Modality Disentanglement](https://mlsanity.com/paper/2104.08223/) (2021)
- [CodeTalker: Speech-Driven 3D Facial Animation with Discrete Motion Prior](https://mlsanity.com/paper/2301.02379/) (2023)
- [Emotional Speech-Driven Animation with Content-Emotion Disentanglement](https://mlsanity.com/paper/2306.08990/) (2023)
- [From Audio to Photoreal Embodiment: Synthesizing Humans in Conversations](https://mlsanity.com/paper/2401.01885/) (2024)
- [FaceDiffuser: Speech-Driven 3D Facial Animation Synthesis Using Diffusion](https://mlsanity.com/paper/2309.11306/) (2023)
- [Responsive Listening Head Generation: A Benchmark Dataset and Baseline](https://mlsanity.com/paper/2112.13548/) (2021)
- [INFP: Audio-Driven Interactive Head Generation in Dyadic Conversations](https://mlsanity.com/paper/2412.04037/) (2024)
- [Dyadic Interaction Modeling for Social Behavior Generation](https://mlsanity.com/paper/2403.09069/) (2024)
- [UniTalker: Scaling up Audio-Driven 3D Facial Animation through A Unified Model](https://mlsanity.com/paper/2408.00762/) (2024)
- [CustomListener: Text-guided Responsive Interaction for User-friendly Listening Head Generation](https://mlsanity.com/paper/2403.00274/) (2024)
- [ReactFace: Online Multiple Appropriate Facial Reaction Generation in Dyadic Interactions](https://mlsanity.com/paper/2305.15748/) (2023)
- [MFR-Net: Multi-faceted Responsive Listening Head Generation via Denoising Diffusion Model](https://mlsanity.com/paper/2308.16635/) (2023)
- [Mimic: Speaking Style Disentanglement for Speech-Driven 3D Facial Animation](https://mlsanity.com/paper/2312.10877/) (2023)
- [Joint Audio-Text Model for Expressive Speech-Driven 3D Facial Animation](https://mlsanity.com/paper/2112.02214/) (2021)
- [KMTalk: Speech-Driven 3D Facial Animation with Key Motion Embedding](https://mlsanity.com/paper/2409.01113/) (2024)
- [ProbTalk3D: Non-Deterministic Emotion Controllable Speech-Driven 3D Facial Animation Synthesis Using VQ-VAE](https://mlsanity.com/paper/2409.07966/) (2024)
- [DEEPTalk: Dynamic Emotion Embedding for Probabilistic Speech-Driven 3D Face Animation](https://mlsanity.com/paper/2408.06010/) (2024)
- [MemoryTalker: Personalized Speech-Driven 3D Facial Animation via Audio-Guided Stylization](https://mlsanity.com/paper/2507.20562/) (2025)
- [GLDiTalker: Speech-Driven 3D Facial Animation with Graph Latent Diffusion Transformer](https://mlsanity.com/paper/2408.01826/) (2024)
- [Wav2Sem: Plug-and-Play Audio Semantic Decoupling for 3D Speech-Driven Facial Animation](https://mlsanity.com/paper/2505.23290/) (2025)
- [DiffusionTalker: Efficient and Compact Speech-Driven 3D Talking Head via Personalizer-Guided Distillation](https://mlsanity.com/paper/2503.18159/) (2025)
- [AdaMesh: Personalized Facial Expressions and Head Poses for Adaptive Speech-Driven 3D Facial Animation](https://mlsanity.com/paper/2310.07236/) (2023)
- [Active Listener: Continuous Generation of Listener's Head Motion Response in Dyadic Interactions](https://mlsanity.com/paper/2409.20188/) (2024)

## Abstract (arXiv)

We propose ECHO for dyadic 3D facial motion generation under a strict dual-stream audio-only setting, formulating the problem as an asymmetric task involving speech-constrained articulation and one-to-many listener reactions. To address this asymmetry, ECHO decomposes motion into a deterministic anchor that captures stable speech-correlated structure and a stochastic residual that models the remaining one-to-many interaction dynamics. On top of this backbone, Motion Memory acts as a training-only regularizer during brief late-stage fine-tuning to provide local priors for weakly conditioned listening windows, while semantic-group scaling controls residual injection across expression, jaw, and neck. This design balances speaking-side articulatory fidelity with listening-side realism and diversity in a single generation process. Results from unified, state-wise, and ablation evaluations show that conversational 3D motion benefits from decomposing stable and uncertain components rather than applying stochasticity uniformly. ECHO provides a practical formulation and technical basis for deployable conversational digital humans under strict audio-only conditions.