# Speech2Properties2Gestures: Gesture-Property Prediction as a Tool for Generating Representational Gestures from Speech

arXiv: https://arxiv.org/abs/2106.14736 · HTML: https://mlsanity.com/paper/2106.14736/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 30 references that are in the map.
- First posted on arXiv 28 Jun 2021.

## Builds on (most cited first, 9 of 9)

- [DistilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter](https://mlsanity.com/paper/1910.01108/) (2019)
- [Speech Gesture Generation from the Trimodal Context of Text, Audio, and Speaker Identity](https://mlsanity.com/paper/2009.02119/) (2020)
- [Robots Learn Social Skills: End-to-End Learning of Co-Speech Gesture Generation for Humanoid Robots](https://mlsanity.com/paper/1810.12541/) (2018)
- [Gesticulator: A framework for semantically-aware speech-driven gesture generation](https://mlsanity.com/paper/2001.09326/) (2020)
- [MoGlow: Probabilistic and controllable motion synthesis using normalising flows](https://mlsanity.com/paper/1905.06598/) (2019)
- [A large, crowdsourced evaluation of gesture generation systems on common data: The GENEA Challenge 2020](https://mlsanity.com/paper/2102.11617/) (2021)
- [Moving fast and slow: Analysis of representations and post-processing in speech-driven automatic gesture generation](https://mlsanity.com/paper/2007.09170/) (2020)
- [Understanding the Predictability of Gesture Parameters from Speech and their Perceptual Importance](https://mlsanity.com/paper/2010.00995/) (2020)
- [Sequence-to-Sequence Predictive Model: From Prosody To Communicative Gestures](https://mlsanity.com/paper/2008.07643/) (2020)

## Built on by (most cited first, 12 of 12)

- [GestureDiffuCLIP: Gesture Diffusion Model with CLIP Latents](https://mlsanity.com/paper/2303.14613/) (2023)
- [ConvoFusion: Multi-Modal Conversational Diffusion for Co-Speech Gesture Synthesis](https://mlsanity.com/paper/2403.17936/) (2024)
- [The GENEA Challenge 2022: A large evaluation of data-driven co-speech gesture generation](https://mlsanity.com/paper/2208.10441/) (2022)
- [Semantic Gesticulator: Semantics-Aware Co-Speech Gesture Synthesis](https://mlsanity.com/paper/2405.09814/) (2024)
- [Rhythmic Gesticulator: Rhythm-Aware Co-Speech Gesture Synthesis with Hierarchical Neural Embeddings](https://mlsanity.com/paper/2210.01448/) (2022)
- [Enabling Synergistic Full-Body Control in Prompt-Based Co-Speech Motion Generation](https://mlsanity.com/paper/2410.00464/) (2024)
- [Evaluating gesture generation in a large-scale open challenge: The GENEA Challenge 2022](https://mlsanity.com/paper/2303.08737/) (2023)
- [Retrieving Semantics from the Deep: an RAG Solution for Gesture Synthesis](https://mlsanity.com/paper/2412.06786/) (2024)
- [Multimodal analysis of the predictability of hand-gesture properties](https://mlsanity.com/paper/2108.05762/) (2021)
- [Vision + Language Applications: A Survey](https://mlsanity.com/paper/2305.14598/) (2023)
- [Social Agent: Mastering Dyadic Nonverbal Behavior Generation via Conversational LLM Agents](https://mlsanity.com/paper/2510.04637/) (2025)
- [Modeling Turn-Taking with Semantically Informed Gestures](https://mlsanity.com/paper/2510.19350/) (2025)

## Abstract (arXiv)

We propose a new framework for gesture generation, aiming to allow data-driven approaches to produce more semantically rich gestures. Our approach first predicts whether to gesture, followed by a prediction of the gesture properties. Those properties are then used as conditioning for a modern probabilistic gesture-generation model capable of high-quality output. This empowers the approach to generate gestures that are both diverse and representational. Follow-ups and more information can be found on the project page: https://svito-zar.github.io/speech2properties2gestures/ .