# Incorporating Spatial Awareness in Data-Driven Gesture Generation for Virtual Agents

arXiv: https://arxiv.org/abs/2408.04127 · HTML: https://mlsanity.com/paper/2408.04127/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 32 references that are in the map.
- First posted on arXiv 7 Aug 2024.

## Builds on (most cited first, 9 of 9)

- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Diffusion-based Generation, Optimization, and Planning in 3D Scenes](https://mlsanity.com/paper/2301.06015/) (2023)
- [BEDLAM: A Synthetic Dataset of Bodies Exhibiting Detailed Lifelike Animated Motion](https://mlsanity.com/paper/2306.16940/) (2023)
- [HUMANISE: Language-conditioned Human Motion Generation in 3D Scenes](https://mlsanity.com/paper/2210.09729/) (2022)
- [Generating 3D People in Scenes without People](https://mlsanity.com/paper/1912.02923/) (2019)
- [Best Practices and Lessons Learned on Synthetic Data](https://mlsanity.com/paper/2404.07503/) (2024)
- [Gesticulator: A framework for semantically-aware speech-driven gesture generation](https://mlsanity.com/paper/2001.09326/) (2020)
- [The GENEA Challenge 2023: A large scale evaluation of gesture generation models in monadic and dyadic settings](https://mlsanity.com/paper/2308.12646/) (2023)
- [Diffusion-Based Co-Speech Gesture Generation Using Joint Text and Audio Representation](https://mlsanity.com/paper/2309.05455/) (2023)

## Built on by (most cited first, 2 of 2)

- [Grounded Gesture Generation: Language, Motion, and Space](https://mlsanity.com/paper/2507.04522/) (2025)
- [InteractGesture: Progressive Chunk Guidance for Continuous Streaming Co-Speech Gesture Control](https://mlsanity.com/paper/2608.25734/) (2026)

## Abstract (arXiv)

This paper focuses on enhancing human-agent communication by integrating spatial context into virtual agents' non-verbal behaviors, specifically gestures. Recent advances in co-speech gesture generation have primarily utilized data-driven methods, which create natural motion but limit the scope of gestures to those performed in a void. Our work aims to extend these methods by enabling generative models to incorporate scene information into speech-driven gesture synthesis. We introduce a novel synthetic gesture dataset tailored for this purpose. This development represents a critical step toward creating embodied conversational agents that interact more naturally with their environment and users.