mlsanityData to 14 Oct 2026

ECHO: Dyadic 3D Facial Motion Generation with Asymmetric Deterministic Articulation and Stochastic Reaction

arXiv 2609.05506 ↗First posted 29 Aug 2026cs.GR cs.CV cs.SDcited by 0

Not on the map. Why?

Abstract (arXiv)

We propose ECHO for dyadic 3D facial motion generation under a strict dual-stream audio-only setting, formulating the problem as an asymmetric task involving speech-constrained articulation and one-to-many listener reactions. To address this asymmetry, ECHO decomposes motion into a deterministic anchor that captures stable speech-correlated structure and a stochastic residual that models the remaining one-to-many interaction dynamics. On top of this backbone, Motion Memory acts as a training-only regularizer during brief late-stage fine-tuning to provide local priors for weakly conditioned listening windows, while semantic-group scaling controls residual injection across expression, jaw, and neck. This design balances speaking-side articulatory fidelity with listening-side realism and diversity in a single generation process. Results from unified, state-wise, and ablation evaluations show that conversational 3D motion benefits from decomposing stable and uncertain components rather than applying stochasticity uniformly. ECHO provides a practical formulation and technical basis for deployable conversational digital humans under strict audio-only conditions.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 26 of its 49 references that are in the map. First posted on arXiv 29 Aug 2026.

Builds on

26 of 49 references.

References · 26

  1. 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,700
  2. 2021HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Unitscited by 2,342
  3. 2019Capture, Learning, and Synthesis of 3D Speaking Stylescited by 259
  4. 2021MeshTalk: 3D Face Animation from Speech using Cross-Modality Disentanglementcited by 190
  5. 2023CodeTalker: Speech-Driven 3D Facial Animation with Discrete Motion Priorcited by 184
  6. 2023Emotional Speech-Driven Animation with Content-Emotion Disentanglementcited by 98
  7. 2024From Audio to Photoreal Embodiment: Synthesizing Humans in Conversationscited by 82
  8. 2023FaceDiffuser: Speech-Driven 3D Facial Animation Synthesis Using Diffusioncited by 78
  9. 2021Responsive Listening Head Generation: A Benchmark Dataset and Baselinecited by 67
  10. 2024INFP: Audio-Driven Interactive Head Generation in Dyadic Conversationscited by 40
  11. 2024Dyadic Interaction Modeling for Social Behavior Generationcited by 32
  12. 2024UniTalker: Scaling up Audio-Driven 3D Facial Animation through A Unified Modelcited by 30
14 more
  1. 2024CustomListener: Text-guided Responsive Interaction for User-friendly Listening Head Generationcited by 22
  2. 2023ReactFace: Online Multiple Appropriate Facial Reaction Generation in Dyadic Interactionscited by 17
  3. 2023MFR-Net: Multi-faceted Responsive Listening Head Generation via Denoising Diffusion Modelcited by 16
  4. 2023Mimic: Speaking Style Disentanglement for Speech-Driven 3D Facial Animationcited by 16
  5. 2021Joint Audio-Text Model for Expressive Speech-Driven 3D Facial Animationcited by 15
  6. 2024KMTalk: Speech-Driven 3D Facial Animation with Key Motion Embeddingcited by 14
  7. 2024ProbTalk3D: Non-Deterministic Emotion Controllable Speech-Driven 3D Facial Animation Synthesis Using VQ-VAEcited by 13
  8. 2024DEEPTalk: Dynamic Emotion Embedding for Probabilistic Speech-Driven 3D Face Animationcited by 12
  9. 2025MemoryTalker: Personalized Speech-Driven 3D Facial Animation via Audio-Guided Stylizationcited by 9
  10. 2024GLDiTalker: Speech-Driven 3D Facial Animation with Graph Latent Diffusion Transformercited by 8
  11. 2025Wav2Sem: Plug-and-Play Audio Semantic Decoupling for 3D Speech-Driven Facial Animationcited by 8
  12. 2025DiffusionTalker: Efficient and Compact Speech-Driven 3D Talking Head via Personalizer-Guided Distillationcited by 4
  13. 2023AdaMesh: Personalized Facial Expressions and Head Poses for Adaptive Speech-Driven 3D Facial Animationcited by 2
  14. 2024Active Listener: Continuous Generation of Listener's Head Motion Response in Dyadic Interactionscited by 1

Built on by

Not cited yet.