# Moving fast and slow: Analysis of representations and post-processing in speech-driven automatic gesture generation

arXiv: https://arxiv.org/abs/2007.09170 · HTML: https://mlsanity.com/paper/2007.09170/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Jul 2020.

## Built on by (most cited first, 20 of 20)

- [Listen, Denoise, Action! Audio-Driven Motion Synthesis with Diffusion Models](https://mlsanity.com/paper/2211.09707/) (2022)
- [BEAT: A Large-Scale Semantic and Emotional Multi-Modal Dataset for Conversational Gestures Synthesis](https://mlsanity.com/paper/2203.05297/) (2022)
- [EMAGE: Towards Unified Holistic Co-Speech Gesture Generation via Expressive Masked Audio Gesture Modeling](https://mlsanity.com/paper/2401.00374/) (2023)
- [A Comprehensive Review of Data-Driven Co-Speech Gesture Generation](https://mlsanity.com/paper/2301.05339/) (2023)
- [Large Motion Model for Unified Multi-Modal Motion Generation](https://mlsanity.com/paper/2404.01284/) (2024)
- [QPGesture: Quantization-Based and Phase-Guided Motion Matching for Natural Speech-Driven Gesture Generation](https://mlsanity.com/paper/2305.11094/) (2023)
- [Evaluating gesture generation in a large-scale open challenge: The GENEA Challenge 2022](https://mlsanity.com/paper/2303.08737/) (2023)
- [Integrated Speech and Gesture Synthesis](https://mlsanity.com/paper/2108.11436/) (2021)
- [Deep Person Generation: A Survey from the Perspective of Face, Pose and Cloth Synthesis](https://mlsanity.com/paper/2109.02081/) (2021)
- [Multimodal analysis of the predictability of hand-gesture properties](https://mlsanity.com/paper/2108.05762/) (2021)
- [Speech2Properties2Gestures: Gesture-Property Prediction as a Tool for Generating Representational Gestures from Speech](https://mlsanity.com/paper/2106.14736/) (2021)
- [The ReprGesture entry to the GENEA Challenge 2022](https://mlsanity.com/paper/2208.12133/) (2022)
- [Motion Generation: A Survey of Generative Approaches and Benchmarks](https://mlsanity.com/paper/2507.05419/) (2025)
- [Towards the generation of synchronized and believable non-verbal facial behaviors of a talking virtual agent](https://mlsanity.com/paper/2311.12804/) (2023)
- [A Framework for Integrating Gesture Generation Models into Interactive Conversational Agents](https://mlsanity.com/paper/2102.12302/) (2021)
- [Learning to Generate Pointing Gestures in Situated Embodied Conversational Agents](https://mlsanity.com/paper/2509.12507/) (2025)
- [Allo-AVA: A Large-Scale Multimodal Conversational AI Dataset for Allocentric Avatar Gesture Animation](https://mlsanity.com/paper/2410.16503/) (2024)
- [Multimodal Quantitative Measures for Multiparty Behaviour Evaluation](https://mlsanity.com/paper/2508.10916/) (2025)
- [Towards Reliable Human Evaluations in Gesture Generation: Insights from a Community-Driven State-of-the-Art Benchmark](https://mlsanity.com/paper/2511.01233/) (2025)
- [Mitigation of gender bias in automatic facial non-verbal behaviors generation](https://mlsanity.com/paper/2410.07274/) (2024)

## Abstract (arXiv)

This paper presents a novel framework for speech-driven gesture production, applicable to virtual agents to enhance human-computer interaction. Specifically, we extend recent deep-learning-based, data-driven methods for speech-driven gesture generation by incorporating representation learning. Our model takes speech as input and produces gestures as output, in the form of a sequence of 3D coordinates. We provide an analysis of different representations for the input (speech) and the output (motion) of the network by both objective and subjective evaluations. We also analyse the importance of smoothing of the produced motion. Our results indicated that the proposed method improved on our baseline in terms of objective measures. For example, it better captured the motion dynamics and better matched the motion-speed distribution. Moreover, we performed user studies on two different datasets. The studies confirmed that our proposed method is perceived as more natural than the baseline, although the difference in the studies was eliminated by appropriate post-processing: hip-centering and smoothing. We conclude that it is important to take both motion representation and post-processing into account when designing an automatic gesture-production method.