mlsanityData to 14 Oct 2026

SalsaAgent: A multimodal embodied language model for interactive dance generation

arXiv 2605.29219 ↗First posted 28 May 2026cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Embodied interaction with humanoids depends on bidirectional nonverbal reactivity, coordination, and synchrony to convey cues and move with a partner. For socially interactive embodied agents, reactive motion generation requires expressive full-body motion that remains contextually appropriate while maintaining spatial and temporal synchrony. We present SalsaAgent, a language model that generates expressive, full-body salsa follower motions in reaction to a human leader and music. We formulate partner interaction as nonverbal token passing, extending the vocabulary of a large language model (LLM) to process discrete motion tokens, pairwise relation tokens, and audio tokens. Our method introduces full-body and pairwise-relation tokenizers, aligns language and motion tokens with automatically derived text descriptions of skeleton dynamics, and applies a two-stage token-to-diffusion pipeline. Subjective and objective evaluations show improved motion quality, two-person spatial coordination, and music and partner coordination relative to prior baselines.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 54 of its 70 references that are in the map. First posted on arXiv 28 May 2026.

Builds on

54 of 70 references.

References · 54

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  4. 2020Denoising Diffusion Implicit Modelscited by 9,781
  5. 2017Neural Discrete Representation Learningcited by 4,712
  6. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,322
  7. 2021Improved Denoising Diffusion Probabilistic Modelscited by 3,761
  8. 2025Wan: Open and Advanced Large-Scale Video Generative Modelscited by 2,708
  9. 2024Gemma 2: Improving Open Language Models at a Practical Sizecited by 1,998
  10. 2023Video-LLaMA: An Instruction-tuned Audio-Visual Language Model for Video Understandingcited by 1,471
  11. 2019Generating Diverse High-Fidelity Images with VQ-VAE-2cited by 1,387
  12. 2023ImageBind: One Embedding Space To Bind Them Allcited by 1,175
38 more
  1. 2022Human Motion Diffusion Modelcited by 1,080
  2. 2023NExT-GPT: Any-to-Any Multimodal LLMcited by 615
  3. 2023T2M-GPT: Generating Human Motion from Textual Descriptions with Discrete Representationscited by 588
  4. 2023MotionGPT: Human Motion as a Foreign Languagecited by 527
  5. 2021Action-Conditioned 3D Human Motion Synthesis with Transformer VAEcited by 501
  6. 2021AI Choreographer: Music Conditioned 3D Dance Generation with AIST++cited by 485
  7. 2020Action2Motion: Conditioned Generation of 3D Human Motionscited by 441
  8. 2020Reliable Fidelity and Diversity Metrics for Generative Modelscited by 434
  9. 2023MoMask: Generative Masked Modeling of 3D Human Motionscited by 400
  10. 2023Human Motion Diffusion as a Generative Priorcited by 357
  11. 2022EDGE: Editable Dance Generation From Musiccited by 287
  12. 2023Pengi: An Audio Language Model for Audio Taskscited by 220
  13. 2022MoFusion: A Framework for Denoising-Diffusion-based Motion Synthesiscited by 218
  14. 2022Bailando: 3D Dance Generation by Actor-Critic GPT with Choreographic Memorycited by 212
  15. 2023InterGen: Diffusion-based Multi-human Motion Generation under Complex Interactionscited by 206
  16. 2023OneLLM: One Framework to Align All Modalities with Languagecited by 187
  17. 2023MotionGPT: Finetuned LLMs Are General-Purpose Motion Generatorscited by 145
  18. 2024WavTokenizer: an Efficient Acoustic Discrete Codec Tokenizer for Audio Language Modelingcited by 129
  19. 2023FineMoGen: Fine-Grained Spatio-Temporal Motion Generation and Editingcited by 106
  20. 2023Inter-X: Towards Versatile Human-Human Interaction Analysiscited by 98
  21. 2022FineDance: A Fine-grained Choreography Dataset for 3D Full Body Dance Generationcited by 88
  22. 2025OmniRetarget: Interaction-Preserving Data Generation for Humanoid Whole-Body Loco-Manipulation and Scene Interactioncited by 71
  23. 2022UDE: A Unified Driving Engine for Human Motion Generationcited by 67
  24. 2023ReMoS: 3D Motion-Conditioned Reaction Synthesis for Two-Person Interactionscited by 62
  25. 2024Duolando: Follower GPT with Off-Policy Reinforcement Learning for Dance Accompanimentcited by 47
  26. 2024Motion-Agent: A Conversational Framework for Human Motion Generation with LLMscited by 47
  27. 2024InterMask: 3D Human Interaction Generation via Collaborative Masked Modelingcited by 47
  28. 2023MusiLingo: Bridging Music and Text with Pre-trained Language Models for Music Captioning and Query Responsecited by 45
  29. 2024FreeMotion: A Unified Framework for Number-free Text-to-Motion Synthesiscited by 42
  30. 2024M3GPT: An Advanced Multimodal, Multitask Framework for Motion Comprehension and Generationcited by 38
  31. 2023Music-Driven Group Choreographycited by 37
  32. 2023Interactive Humanoid: Online Full-Body Motion Reaction Synthesis with Social Affordance Canonicalization and Forecastingcited by 27
  33. 2025A Survey on Human Interaction Motion Generationcited by 25
  34. 2024InterDance:Reactive 3D Dance Generation with Realistic Duet Interactionscited by 22
  35. 2023MotionScript: Natural Language Descriptions for Expressive 3D Human Motionscited by 21
  36. 2024SOLAMI: Social Vision-Language-Action Modeling for Immersive Interaction with 3D Autonomous Characterscited by 18
  37. 2023Controllable Group Choreography using Contrastive Diffusioncited by 17
  38. 2025SocialGen: Modeling Multi-Human Social Interaction with Language Modelscited by 14

50 most cited of 54 All 54 (JSON)

Built on by

Not cited yet.