# A Review of Evaluation Practices of Gesture Generation in Embodied Conversational Agents

arXiv: https://arxiv.org/abs/2101.03769 · HTML: https://mlsanity.com/paper/2101.03769/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Jan 2021.

## Built on by (most cited first, 24 of 24)

- [A Comprehensive Review of Data-Driven Co-Speech Gesture Generation](https://mlsanity.com/paper/2301.05339/) (2023)
- [The GENEA Challenge 2022: A large evaluation of data-driven co-speech gesture generation](https://mlsanity.com/paper/2208.10441/) (2022)
- [EmotionGesture: Audio-Driven Diverse Emotional Co-Speech 3D Gesture Generation](https://mlsanity.com/paper/2305.18891/) (2023)
- [Rhythmic Gesticulator: Rhythm-Aware Co-Speech Gesture Synthesis with Hierarchical Neural Embeddings](https://mlsanity.com/paper/2210.01448/) (2022)
- [Evaluating gesture generation in a large-scale open challenge: The GENEA Challenge 2022](https://mlsanity.com/paper/2303.08737/) (2023)
- [Retrieving Semantics from the Deep: an RAG Solution for Gesture Synthesis](https://mlsanity.com/paper/2412.06786/) (2024)
- [DiffMotion: Speech-Driven Gesture Synthesis Using Denoising Diffusion Model](https://mlsanity.com/paper/2301.10047/) (2023)
- [Diverse 3D Hand Gesture Prediction from Body Dynamics by Bilateral Hand Disentanglement](https://mlsanity.com/paper/2303.01765/) (2023)
- [The DiffuseStyleGesture+ entry to the GENEA Challenge 2023](https://mlsanity.com/paper/2308.13879/) (2023)
- [Didn't see that coming: a survey on non-verbal social human behavior forecasting](https://mlsanity.com/paper/2203.02480/) (2022)
- [Zero-Shot Style Transfer for Gesture Animation driven by Text and Speech using Adversarial Disentanglement of Multimodal Style Encoding](https://mlsanity.com/paper/2208.01917/) (2022)
- [Speech-driven Personalized Gesture Synthetics: Harnessing Automatic Fuzzy Feature Inference](https://mlsanity.com/paper/2403.10805/) (2024)
- [HOP: Heterogeneous Topology-based Multimodal Entanglement for Co-Speech Gesture Generation](https://mlsanity.com/paper/2503.01175/) (2025)
- [Tachikuma: Understading Complex Interactions with Multi-Character and Novel Objects by Large Language Models](https://mlsanity.com/paper/2307.12573/) (2023)
- [Towards the generation of synchronized and believable non-verbal facial behaviors of a talking virtual agent](https://mlsanity.com/paper/2311.12804/) (2023)
- [The Importance of Multimodal Emotion Conditioning and Affect Consistency for Embodied Conversational Agents](https://mlsanity.com/paper/2309.15311/) (2023)
- [Towards a GENEA Leaderboard -- an Extended, Living Benchmark for Evaluating and Advancing Conversational Motion Synthesis](https://mlsanity.com/paper/2410.06327/) (2024)
- [Speech-Gesture GAN: Gesture Generation for Robots and Embodied Agents](https://mlsanity.com/paper/2309.09346/) (2023)
- [Multimodal Quantitative Measures for Multiparty Behaviour Evaluation](https://mlsanity.com/paper/2508.10916/) (2025)
- [Talking Surveys: How Photorealistic Embodied Conversational Agents Shape Response Quality, Engagement, and Satisfaction](https://mlsanity.com/paper/2508.02376/) (2025)
- [Gesture Evaluation in Virtual Reality](https://mlsanity.com/paper/2509.12816/) (2025)
- [Towards Reliable Human Evaluations in Gesture Generation: Insights from a Community-Driven State-of-the-Art Benchmark](https://mlsanity.com/paper/2511.01233/) (2025)
- [Audio is all in one: speech-driven gesture synthetics using WavLM pre-trained model](https://mlsanity.com/paper/2308.05995/) (2023)
- [2D or not 2D: How Does the Dimensionality of Gesture Representation Affect 3D Co-Speech Gesture Generation?](https://mlsanity.com/paper/2409.10357/) (2024)

## Abstract (arXiv)

Embodied conversational agents (ECA) are often designed to produce nonverbal behavior to complement or enhance their verbal communication. One such form of nonverbal behavior is co-speech gesturing, which involves movements that the agent makes with its arms and hands that are paired with verbal communication. Co-speech gestures for ECAs can be created using different generation methods, divided into rule-based and data-driven processes, with the latter gaining traction because of the increasing interest from the applied machine learning community. However, reports on gesture generation methods use a variety of evaluation measures, which hinders comparison. To address this, we present a systematic review on co-speech gesture generation methods for iconic, metaphoric, deictic, and beat gestures, including reported evaluation methods. We review 22 studies that have an ECA with a human-like upper body that uses co-speech gesturing in social human-agent interaction. This includes studies that use human participants to evaluate performance. We found most studies use a within-subject design and rely on a form of subjective evaluation, but without a systematic approach. We argue that the field requires more rigorous and uniform tools for co-speech gesture evaluation, and formulate recommendations for empirical evaluation, including standardized phrases and example scenarios to help systematically test generative models across studies. Furthermore, we also propose a checklist that can be used to report relevant information for the evaluation of generative models, as well as to evaluate co-speech gesture use.