mlsanityData to 7 Oct 2026

AnyMo: Scaling Any-Modality Conditional Motion Generation with Masked Modeling

arXiv 2605.29488 ↗First posted 28 May 2026cs.CV cs.AIcited by 1

Not on the map. Why?

Abstract (arXiv)

Conditional human motion generation remains a fundamental challenge in computer vision and robotics. Despite significant progress, current methods are often constrained by fixed modality configurations and task-specific architectures, leaving cross-modal interactions and the scaling laws of multimodal-conditioned synthesis largely underexplored. A key bottleneck is the scarcity of large-scale modality-aligned motion data, limiting generalization across diverse control signals. In this work, we introduce OmniHuMo, a large-scale, high-quality dataset comprising over 5,000 hours of motion and 3.2 million sequences with precisely aligned multimodal annotations (e.g., text, speech, music, and trajectory). Leveraging OmniHuMo, we propose AnyMo, a unified multimodal framework combining a Residual FSQ-based motion tokenizer with a scalable masked modeling transformer, enabling high-quality motion synthesis under arbitrary modality combinations. Extensive experiments show that AnyMo achieves high-fidelity synthesis while offering flexible control over both spatial and stylistic attributes.

Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 56 of its 65 references that are in the map. First posted on arXiv 28 May 2026.

Builds on

56 of 65 references.

References · 56

  1. 2017Attention Is All You Needcited by 59,932
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  4. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  5. 2017Neural Discrete Representation Learningcited by 4,650
  6. 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 3,972
  7. 2022Scaling Instruction-Finetuned Language Modelscited by 2,997
  8. 2025Qwen3-VL Technical Reportcited by 2,534
  9. 2019AMASS: Archive of Motion Capture as Surface Shapescited by 1,208
  10. 2022MaskGIT: Masked Generative Image Transformercited by 1,082
  11. 2021DROID-SLAM: Deep Visual SLAM for Monocular, Stereo, and RGB-D Camerascited by 673
  12. 2023T2M-GPT: Generating Human Motion from Textual Descriptions with Discrete Representationscited by 577
38 more
  1. 2022TEMOS: Generating diverse human motions from textual descriptionscited by 486
  2. 2021AI Choreographer: Music Conditioned 3D Dance Generation with AIST++cited by 479
  3. 2023Finite Scalar Quantization: VQ-VAE Made Simplecited by 425
  4. 2024YOLOv11: An Overview of the Key Architectural Enhancementscited by 414
  5. 2023MoMask: Generative Masked Modeling of 3D Human Motionscited by 388
  6. 2016The KIT Motion-Language Datasetcited by 334
  7. 2022Exploring Video Quality Assessment on User Generated Contents from Aesthetic and Technical Perspectivescited by 314
  8. 2022EDGE: Editable Dance Generation From Musiccited by 284
  9. 2025LLaDA 1.5: Variance-Reduced Preference Optimization for Large Language Diffusion Modelscited by 273
  10. 2022Unifying Flow, Stereo and Depth Estimationcited by 270
  11. 2023OmniControl: Control Any Joint at Any Time for Human Motion Generationcited by 233
  12. 2022Bailando: 3D Dance Generation by Actor-Critic GPT with Choreographic Memorycited by 210
  13. 2022Generating Holistic 3D Human Motion from Speechcited by 180
  14. 2020TransNet V2: An effective deep network architecture for fast shot transition detectioncited by 166
  15. 2023SMPLer-X: Scaling Up Expressive Human Pose and Shape Estimationcited by 154
  16. 2022MOTRv2: Bootstrapping End-to-End Multi-Object Tracking by Pretrained Object Detectorscited by 148
  17. 2025LLaDA-V: Large Language Diffusion Models with Visual Instruction Tuningcited by 145
  18. 2022Hybrid Transformers for Music Source Separationcited by 134
  19. 2023MMM: Generative Masked Motion Modelcited by 129
  20. 2024World-Grounded Human Motion Recovery via Gravity-View Coordinatescited by 127
  21. 2024WavTokenizer: an Efficient Acoustic Discrete Codec Tokenizer for Audio Language Modelingcited by 124
  22. 2023EMAGE: Towards Unified Holistic Co-Speech Gesture Generation via Expressive Masked Audio Gesture Modelingcited by 101
  23. 2023TLControl: Trajectory and Language Control for Human Motion Synthesiscited by 93
  24. 2025MotionStreamer: Streaming Motion Generation via Diffusion-based Autoregressive Model in Causal Latent Spacecited by 82
  25. 2024OpenHumanVid: A Large-Scale High-Quality Dataset for Enhancing Human-Centric Video Generationcited by 77
  26. 2025RF-DETR: Neural Architecture Search for Real-Time Detection Transformerscited by 70
  27. 2025Go to Zero: Towards Zero-shot Motion Generation with Million-scale Datacited by 67
  28. 2024BAMM: Bidirectional Autoregressive Motion Modelcited by 54
  29. 2024ScaMo: Exploring the Scaling Law in Autoregressive Motion Generation Modelcited by 52
  30. 2024Generating Human Motion in 3D Scenes from Text Descriptionscited by 49
  31. 2024The Language of Motion: Unifying Verbal and Non-verbal Language of 3D Human Motioncited by 46
  32. 2025SnapMoGen: Human Motion Generation from Expressive Textscited by 44
  33. 2025GENMO: A GENeralist Model for Human MOtioncited by 43
  34. 2025Motion Anything: Any to Motion Generationcited by 40
  35. 2024MaskControl: Spatio-Temporal Control for Masked Motion Synthesiscited by 37
  36. 2026Kimodo: Scaling Controllable Human Motion Generationcited by 33
  37. 2025HY-Motion 1.0: Scaling Flow Matching Models for Text-To-Motion Generationcited by 32
  38. 2025Motion-X++: A Large-Scale Multimodal 3D Whole-body Human Motion Datasetcited by 30

50 most cited of 56 All 56 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2026: 1.

1 paper.