mlsanityData to 14 Oct 2026

ECHO-G: Embodied Co-speech Humanoid mOtion Generation

arXiv 2609.39575 ↗First posted 30 Sep 2026cs.RO cs.AIcited by 0

On the map: Diffusion models → Audio, speech and music

Abstract (arXiv)

Generating full-body co-speech motion for humanoid robots requires coordinating speech prosody, linguistic content, and embodiment-specific motion. To this end, we present ECHO-G, a framework jointly conditioned on speech audio and timed transcripts. Its Speech-Grounded Diffusion Transformer (SGDiT) combines frame-aligned acoustic features with token-level linguistic context, preserving their distinct granularities. Trained with rectified flow matching, it models one-to-many utterance-motion relationships directly in robot space. To support training and evaluation, we introduce a BEAT2-derived audio-text-robot dataset and a benchmark covering co-speech characteristics, robot-motion quality, and runtime efficiency. Comparative evaluation supports direct robot-space generation over the evaluated human-motion generation and retargeting pipelines, while modality ablations highlight the benefits of joint audio-text conditioning. We further demonstrate deployment on a physical humanoid robot. A complementary video-rating study also favors joint conditioning over the alternatives. The dataset and training, inference, and evaluation code are available through our project page.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Audio, speech and music, one of 14 directions in Diffusion models. Builds on 23 of its 26 references that are in the map. First posted on arXiv 30 Sep 2026.

Builds on

23 of 26 references.

Closest ancestors · 2

Same direction, newest first

  1. 2026RoboGesture: Real-Time Semantic-aligned Co-Speech Gestures Generation for Humanoid Interaction27 Aug
  2. 2025GestureLSM: Latent Shortcut based Co-Speech Gesture Generation with Spatial-Temporal Modeling31 Jan

Other references · 21

  1. 2022Scalable Diffusion Models with Transformerscited by 6,054
  2. 2022Flow Matching for Generative Modelingcited by 5,702
  3. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,905
  4. 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,700
  5. 2021AI Choreographer: Music Conditioned 3D Dance Generation with AIST++cited by 485
  6. 2020Speech Gesture Generation from the Trimodal Context of Text, Audio, and Speaker Identitycited by 202
  7. 2022BEAT: A Large-Scale Semantic and Emotional Multi-Modal Dataset for Conversational Gestures Synthesiscited by 178
  8. 2018Robots Learn Social Skills: End-to-End Learning of Co-Speech Gesture Generation for Humanoid Robotscited by 110
  9. 2021Audio2Gestures: Generating Diverse Gestures from Speech Audio with Conditional Variational Autoencoderscited by 109
  10. 2023EMAGE: Towards Unified Holistic Co-Speech Gesture Generation via Expressive Masked Audio Gesture Modelingcited by 105
  11. 2025SONIC: Supersizing Motion Tracking for Natural Humanoid Whole-Body Controlcited by 101
  12. 2025Retargeting Matters: General Motion Retargeting for Humanoid Motion Trackingcited by 75
9 more
  1. 2025OmniRetarget: Interaction-Preserving Data Generation for Humanoid Whole-Body Loco-Manipulation and Scene Interactioncited by 71
  2. 2024DiffSHEG: A Diffusion-Based Approach for Real-Time Speech-driven Holistic 3D Expression and Gesture Generationcited by 62
  3. 2026TextOp: Real-time Interactive Text-Driven Humanoid Robot Motion Generation and Controlcited by 14
  4. 2026FRoM-W1: Towards General Humanoid Whole-Body Control with Language Instructionscited by 10
  5. 2026HoloMotion-1 Technical Reportcited by 8
  6. 2025Do You Have Freestyle? Expressive Humanoid Locomotion via Audio Controlcited by 7
  7. 2025CoordSpeaker: Exploiting Gesture Captioning for Coordinated Caption-Empowered Co-Speech Gesture Generationcited by 4
  8. 2026PhysDrift: Bridging the Embodiment Gap in Humanoid Co-Speech Motion Generationcited by 3
  9. 2026OMG: Omni-Modal Motion Generation for Generalist Humanoid Controlcited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Semantic Co-Speech Gesture Synthesis and Real-Time Control for Humanoid Robots ↗
  2. 2025Commanding Humanoid by Free-form Language: A Large Language Action Model with Unified Motion Vocabulary
  3. 2026Co-Speech with You: Training-Free Personalization of Robot Co-Speech Gestures
  4. 2024Harmon: Whole-Body Motion Generation of Humanoid Robots from Language Descriptions
  5. 2026MIRA: Real-Time Full-Duplex Human-Robot Interaction for Embodied Companions ↗
  6. 2026ECHO: Edge-Cloud Humanoid Orchestration for Language-to-Motion Control
  7. 2026Puppeteer: Object-Grounded Posture-Aware Co-Speech Gesture Generation
  8. 2026Speech2Grasp: Data-Efficient Transfer of Text-Conditioned Grasp Detection to Speech in Humanoid Robots