mlsanityData to 14 Oct 2026

SICAGE: Speaker-Independent Culture-Aware Gesture Generation using TED4C-L Dataset

arXiv 2606.30001 ↗First posted 29 Jun 2026cs.CV cs.GR cs.HCcited by 0

Not on the map. Why?

Abstract (arXiv)

Recent co-speech gesture generation methods often overlook cultural differences, limiting their effectiveness in human-agent interaction. Moreover, culture-conditioned models are rarely evaluated under speaker-disjoint splits, so apparent "cultural" behavior may be confounded with speaker-specific gesturing style. We introduce SICAGE, a modular framework for culture-aware co-speech gesture generation that conditions motion synthesis models on speaker-independent cultural representations. SICAGE learns these representations from audio and text by treating each speaker as a separate domain while imposing invariance across speakers. This encourages representations to remain culture-discriminative while reducing dependence on speaker identity. The resulting cultural embeddings condition a multimodal generator to produce culturally appropriate gestures. We instantiate this idea with two domain generalization approaches: adversarial learning and Fishr regularization. We further introduce ALaDiT, a real-time diffusion-based gesture generator designed to efficiently incorporate the learned cultural embeddings. To validate our method, we built TED4C-L, a 106-hour multimodal dataset of 764 TED speakers from four cultural groups. Experiments show that SICAGE improves motion realism, diversity, beat synchronization, semantic relevance, and cultural consistency.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 35 of its 66 references that are in the map. First posted on arXiv 29 Jun 2026.

Builds on

35 of 66 references.

References · 35

  1. 2017Attention Is All You Needcited by 60,361
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2020Denoising Diffusion Probabilistic Modelscited by 21,076
  4. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 5,045
  5. 2017Neural Discrete Representation Learningcited by 4,712
  6. 2020Supervised Contrastive Learningcited by 2,829
  7. 2017Arbitrary Style Transfer in Real-time with Adaptive Instance Normalizationcited by 2,010
  8. 2014Unsupervised Domain Adaptation by Backpropagationcited by 1,917
  9. 2018On the Continuity of Rotation Representations in Neural Networkscited by 1,214
  10. 2018OpenPose: Realtime Multi-Person 2D Pose Estimation using Part Affinity Fieldscited by 1,162
  11. 2022Human Motion Diffusion Modelcited by 1,080
  12. 2021Generalizing to Unseen Domains: A Survey on Domain Generalizationcited by 766
23 more
  1. 2020Language-agnostic BERT Sentence Embeddingcited by 627
  2. 2020Unsupervised Cross-lingual Representation Learning for Speech Recognitioncited by 501
  3. 2021AI Choreographer: Music Conditioned 3D Dance Generation with AIST++cited by 485
  4. 2020Learning to Learn Single Domain Generalizationcited by 296
  5. 2020Speech Gesture Generation from the Trimodal Context of Text, Audio, and Speaker Identitycited by 202
  6. 2021Fishr: Invariant Gradient Variances for Out-of-Distribution Generalizationcited by 186
  7. 2022BEAT: A Large-Scale Semantic and Emotional Multi-Modal Dataset for Conversational Gestures Synthesiscited by 178
  8. 2016Panoptic Studio: A Massively Multiview System for Social Interaction Capturecited by 173
  9. 2019Domain Generalization Using a Mixture of Multiple Latent Domainscited by 170
  10. 2021FSDR: Frequency Space Domain Randomization for Domain Generalizationcited by 164
  11. 2023GestureDiffuCLIP: Gesture Diffusion Model with CLIP Latentscited by 158
  12. 2023Taming Diffusion Models for Audio-Driven Co-Speech Gesture Generationcited by 129
  13. 2018Robots Learn Social Skills: End-to-End Learning of Co-Speech Gesture Generation for Humanoid Robotscited by 110
  14. 2020Learning to Learn with Variational Information Bottleneck for Domain Generalizationcited by 102
  15. 2023DiffuseStyleGesture: Stylized Audio-Driven Co-Speech Gesture Generation with Diffusion Modelscited by 85
  16. 2023QPGesture: Quantization-Based and Phase-Guided Motion Matching for Natural Speech-Driven Gesture Generationcited by 40
  17. 2023The DiffuseStyleGesture+ entry to the GENEA Challenge 2023cited by 14
  18. 2022EnCBP: A New Benchmark Dataset for Finer-Grained Cultural Background Prediction in Englishcited by 7
  19. 2024MDT-A2G: Exploring Masked Diffusion Transformers for Co-Speech Gesture Generationcited by 7
  20. 2022Co-Located Human-Human Interaction Analysis using Nonverbal Cues: A Surveycited by 6
  21. 2023GesGPT: Speech Gesture Synthesis With Text Parsing from ChatGPTcited by 6
  22. 2023Large language models in textual analysis for gesture selectioncited by 6
  23. 2024Labeling Sentences with Symbolic and Deictic Gestures via Semantic Similaritycited by 2

Built on by

Not cited yet.