# Efficient Emotion-Aware Iconic Gesture Prediction for Robot Co-Speech

arXiv: https://arxiv.org/abs/2604.11417 · HTML: https://mlsanity.com/paper/2604.11417/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 39 references that are in the map.
- First posted on arXiv 13 Apr 2026.

## Builds on (most cited first, 20 of 20)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://mlsanity.com/paper/1908.10084/) (2019)
- [BEAT: A Large-Scale Semantic and Emotional Multi-Modal Dataset for Conversational Gestures Synthesis](https://mlsanity.com/paper/2203.05297/) (2022)
- [Robots Learn Social Skills: End-to-End Learning of Co-Speech Gesture Generation for Humanoid Robots](https://mlsanity.com/paper/1810.12541/) (2018)
- [EMAGE: Towards Unified Holistic Co-Speech Gesture Generation via Expressive Masked Audio Gesture Modeling](https://mlsanity.com/paper/2401.00374/) (2023)
- [DiffuseStyleGesture: Stylized Audio-Driven Co-Speech Gesture Generation with Diffusion Models](https://mlsanity.com/paper/2305.04919/) (2023)
- [Gesticulator: A framework for semantically-aware speech-driven gesture generation](https://mlsanity.com/paper/2001.09326/) (2020)
- [A Comprehensive Review of Data-Driven Co-Speech Gesture Generation](https://mlsanity.com/paper/2301.05339/) (2023)
- [SemTalk: Holistic Co-speech Motion Generation with Frame-level Semantic Emphasis](https://mlsanity.com/paper/2412.16563/) (2024)
- [Synthetic Thermal and RGB Videos for Automatic Pain Assessment utilizing a Vision-MLP Architecture](https://mlsanity.com/paper/2407.19811/) (2024)
- [Emo2Vec: Learning Generalized Emotion Representation by Multi-task Training](https://mlsanity.com/paper/1809.04505/) (2018)
- [Efficient Pain Recognition via Respiration Signals: A Single Cross-Attention Transformer Multi-Window Fusion Pipeline](https://mlsanity.com/paper/2507.21886/) (2025)
- [Meaning and understanding in large language models](https://mlsanity.com/paper/2310.17407/) (2023)
- [A View on Edge caching Applications](https://mlsanity.com/paper/1907.12359/) (2019)
- [Data Augmentation for 3DMM-based Arousal-Valence Prediction for HRI](https://mlsanity.com/paper/2410.00349/) (2024)
- [Speech-Gesture GAN: Gesture Generation for Robots and Embodied Agents](https://mlsanity.com/paper/2309.09346/) (2023)
- [SARGes: Semantically Aligned Reliable Gesture Generation via Intent Chain](https://mlsanity.com/paper/2503.20202/) (2025)
- [Co-Speech Gesture and Facial Expression Generation for Non-Photorealistic 3D Characters](https://mlsanity.com/paper/2506.16159/) (2025)

## Built on by (most cited first, 4 of 4)

- [Towards a Unified Modality-Agnostic Multimodal Framework for Cognitive Workload Assessment](https://mlsanity.com/paper/2609.20199/) (2026)
- [One-Block Transformer (1BT) for EEG-Based Cognitive Workload Assessment](https://mlsanity.com/paper/2605.00856/) (2026)
- [Efficient and Interpretable Body-Based Emotion Recognition with Lightweight Temporal Convolutional Networks](https://mlsanity.com/paper/2607.20820/) (2026)
- [MUPA2E: Multimodal Unified Perception with Asymmetric Attention for Emotion Assessment](https://mlsanity.com/paper/2608.15999/) (2026)

## Abstract (arXiv)

Co-speech gestures increase engagement and improve speech understanding. Most data-driven robot systems generate rhythmic beat-like motion, yet few integrate semantic emphasis. To address this, we propose a lightweight transformer that derives iconic gesture placement and intensity from text and emotion alone, requiring no audio input at inference time. The model outperforms GPT-4o in both semantic gesture placement classification and intensity regression on the BEAT2 dataset, while remaining computationally compact and suitable for real-time deployment on embodied agents.