# Deep Gesture Generation for Social Robots Using Type-Specific Libraries

arXiv: https://arxiv.org/abs/2210.06790 · HTML: https://mlsanity.com/paper/2210.06790/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 13 Oct 2022.

## Built on by (most cited first, 5 of 5)

- [Evaluating gesture generation in a large-scale open challenge: The GENEA Challenge 2022](https://mlsanity.com/paper/2303.08737/) (2023)
- [GesGPT: Speech Gesture Synthesis With Text Parsing from ChatGPT](https://mlsanity.com/paper/2303.13013/) (2023)
- [Towards a GENEA Leaderboard -- an Extended, Living Benchmark for Evaluating and Advancing Conversational Motion Synthesis](https://mlsanity.com/paper/2410.06327/) (2024)
- [SARGes: Semantically Aligned Reliable Gesture Generation via Intent Chain](https://mlsanity.com/paper/2503.20202/) (2025)
- [Robot guide with multi-agent control and automatic scenario generation with LLM](https://mlsanity.com/paper/2509.10317/) (2025)

## Abstract (arXiv)

Body language such as conversational gesture is a powerful way to ease communication. Conversational gestures do not only make a speech more lively but also contain semantic meaning that helps to stress important information in the discussion. In the field of robotics, giving conversational agents (humanoid robots or virtual avatars) the ability to properly use gestures is critical, yet remain a task of extraordinary difficulty. This is because given only a text as input, there are many possibilities and ambiguities to generate an appropriate gesture. Different to previous works we propose a new method that explicitly takes into account the gesture types to reduce these ambiguities and generate human-like conversational gestures. Key to our proposed system is a new gesture database built on the TED dataset that allows us to map a word to one of three types of gestures: "Imagistic" gestures, which express the content of the speech, "Beat" gestures, which emphasize words, and "No gestures." We propose a system that first maps the words in the input text to their corresponding gesture type, generate type-specific gestures and combine the generated gestures into one final smooth gesture. In our comparative experiments, the effectiveness of the proposed method was confirmed in user studies for both avatar and humanoid robot.