# Body of Her: A Preliminary Study on End-to-End Humanoid Agent

arXiv: https://arxiv.org/abs/2408.02879 · HTML: https://mlsanity.com/paper/2408.02879/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- Placed in Multimodal and embodied agents, one of 16 directions in LLM agents and tool use.
- Builds on 14 of its 33 references that are in the map.
- First posted on arXiv 6 Aug 2024.

## Placed in

- LLM agents and tool use / [Multimodal and embodied agents](https://mlsanity.com/agents/multimodal-and-embodied-agents/)

## Builds on (most cited first, 14 of 14)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Scaling Rectified Flow Transformers for High-Resolution Image Synthesis](https://mlsanity.com/paper/2403.03206/) (2024)
- [Learning to summarize from human feedback](https://mlsanity.com/paper/2009.01325/) (2020)
- [Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasets](https://mlsanity.com/paper/2311.15127/) (2023)
- [Scaling Vision Transformers to 22 Billion Parameters](https://mlsanity.com/paper/2302.05442/) (2023)
- [Autoregressive Image Generation without Vector Quantization](https://mlsanity.com/paper/2406.11838/) (2024)
- [GigaSpeech: An Evolving, Multi-domain ASR Corpus with 10,000 Hours of Transcribed Audio](https://mlsanity.com/paper/2106.06909/) (2021)
- [Effective Whole-body Pose Estimation with Two-stages Distillation](https://mlsanity.com/paper/2307.15880/) (2023)
- [An Image is Worth 32 Tokens for Reconstruction and Generation](https://mlsanity.com/paper/2406.07550/) (2024)
- [Query-Key Normalization for Transformers](https://mlsanity.com/paper/2010.04245/) (2020)
- [GestureDiffuCLIP: Gesture Diffusion Model with CLIP Latents](https://mlsanity.com/paper/2303.14613/) (2023)
- [SpeechGPT-Gen: Scaling Chain-of-Information Speech Generation](https://mlsanity.com/paper/2401.13527/) (2024)
- [SPA: Verbal Interactions between Agents and Avatars in Shared Virtual Environments using Propositional Planning](https://mlsanity.com/paper/2002.03246/) (2020)

## Built on by (most cited first, 15 of 15)

- [HunyuanVideo-Avatar: High-Fidelity Audio-Driven Human Animation for Multiple Characters](https://mlsanity.com/paper/2505.20156/) (2025)
- [The Dawn of Video Generation: Preliminary Explorations with SORA-like Models](https://mlsanity.com/paper/2410.05227/) (2024)
- [SpeakerVid-5M: A Large-Scale High-Quality Dataset for Audio-Visual Dyadic Interactive Human Generation](https://mlsanity.com/paper/2507.09862/) (2025)
- [MIDAS: Multimodal Interactive Digital-humAn Synthesis via Real-time Autoregressive Video Generation](https://mlsanity.com/paper/2508.19320/) (2025)
- [SOLAMI: Social Vision-Language-Action Modeling for Immersive Interaction with 3D Autonomous Characters](https://mlsanity.com/paper/2412.00174/) (2024)
- [X-Streamer: Unified Human World Modeling with Audiovisual Interaction](https://mlsanity.com/paper/2509.21574/) (2025)
- [SViMo: Synchronized Diffusion for Video and Motion Generation in Hand-object Interaction Scenarios](https://mlsanity.com/paper/2506.02444/) (2025)
- [FlowAct-R1: Towards Interactive Humanoid Video Generation](https://mlsanity.com/paper/2601.10103/) (2026)
- [Wan-Streamer v0.1: End-to-end Real-time Interactive Foundation Models](https://mlsanity.com/paper/2606.25041/) (2026)
- [OmniMotion-X: Versatile Multimodal Whole-Body Motion Generation](https://mlsanity.com/paper/2510.19789/) (2025)
- [MAViD: A Multimodal Framework for Audio-Visual Dialogue Understanding and Generation](https://mlsanity.com/paper/2512.03034/) (2025)
- [ProAct: Harnessing Streaming Motion Generation and Agentic Reasoning for Real-Time Embodied Social Interaction](https://mlsanity.com/paper/2602.14048/) (2026)
- [Live Avatar: Streaming Real-time Audio-Driven Avatar Generation with Infinite Length](https://mlsanity.com/paper/2512.04677/) (2025)
- [Wan-Streamer v0.2: Higher Resolution, Same Latency](https://mlsanity.com/paper/2607.04443/) (2026)
- [Video = World + Event Stream](https://mlsanity.com/paper/2607.15038/) (2026)

## Abstract (arXiv)

Interactive virtual humanoid agent is a crucial interface with the physical world. A relatively complete humanoid agent first needs to have face and body, then possess both verbal and non-verbal (such as eye contact, facial expression, lip motion, gesture, and manipulation) abilities, and finally, it is capable of real-time duplex communication, e.g., the ability to actively interrupt conversations. Most prior systems typically only consider a subset of these elements, leaving a gap from realistic humanoid agent. In this work, we propose a real-time, duplex, interactive end-to-end network capable of modeling realistic agent behaviors, including speech, full-body movements for talking, responding, idling, and manipulation. This system is a multimodal model integrating audio and visual inputs, extended from a pre-trained large language model (LLM). We collect approximately 200,000 hours of audio, around 130,000 hours of video data, and about 20,000 alignment samples to build the model. The final model demonstrates capabilities that are difficult to achieve in previous systems, such as generalized object manipulation. This work performs a preliminary exploration of the end-to-end approach in this field, aiming to inspire further research towards scaling up.