AnyMo: Scaling Any-Modality Conditional Motion Generation with Masked Modeling
Not on the map. Why?
Abstract (arXiv)
Conditional human motion generation remains a fundamental challenge in computer vision and robotics. Despite significant progress, current methods are often constrained by fixed modality configurations and task-specific architectures, leaving cross-modal interactions and the scaling laws of multimodal-conditioned synthesis largely underexplored. A key bottleneck is the scarcity of large-scale modality-aligned motion data, limiting generalization across diverse control signals. In this work, we introduce OmniHuMo, a large-scale, high-quality dataset comprising over 5,000 hours of motion and 3.2 million sequences with precisely aligned multimodal annotations (e.g., text, speech, music, and trajectory). Leveraging OmniHuMo, we propose AnyMo, a unified multimodal framework combining a Residual FSQ-based motion tokenizer with a scalable masked modeling transformer, enabling high-quality motion synthesis under arbitrary modality combinations. Extensive experiments show that AnyMo achieves high-fidelity synthesis while offering flexible control over both spatial and stylistic attributes.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 56 of its 65 references that are in the map. First posted on arXiv 28 May 2026.
Builds on
56 of 65 references.
References · 56
- 2017Attention Is All You Needcited by 59,932
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
- 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
- 2017Neural Discrete Representation Learningcited by 4,650
- 2022Robust Speech Recognition via Large-Scale Weak Supervisioncited by 3,972
- 2022Scaling Instruction-Finetuned Language Modelscited by 2,997
- 2025Qwen3-VL Technical Reportcited by 2,534
- 2019AMASS: Archive of Motion Capture as Surface Shapescited by 1,208
- 2022MaskGIT: Masked Generative Image Transformercited by 1,082
- 2021DROID-SLAM: Deep Visual SLAM for Monocular, Stereo, and RGB-D Camerascited by 673
- 2023T2M-GPT: Generating Human Motion from Textual Descriptions with Discrete Representationscited by 577
38 more
- 2022TEMOS: Generating diverse human motions from textual descriptionscited by 486
- 2021AI Choreographer: Music Conditioned 3D Dance Generation with AIST++cited by 479
- 2023Finite Scalar Quantization: VQ-VAE Made Simplecited by 425
- 2024YOLOv11: An Overview of the Key Architectural Enhancementscited by 414
- 2023MoMask: Generative Masked Modeling of 3D Human Motionscited by 388
- 2016The KIT Motion-Language Datasetcited by 334
- 2022Exploring Video Quality Assessment on User Generated Contents from Aesthetic and Technical Perspectivescited by 314
- 2022EDGE: Editable Dance Generation From Musiccited by 284
- 2025LLaDA 1.5: Variance-Reduced Preference Optimization for Large Language Diffusion Modelscited by 273
- 2022Unifying Flow, Stereo and Depth Estimationcited by 270
- 2023OmniControl: Control Any Joint at Any Time for Human Motion Generationcited by 233
- 2022Bailando: 3D Dance Generation by Actor-Critic GPT with Choreographic Memorycited by 210
- 2022Generating Holistic 3D Human Motion from Speechcited by 180
- 2020TransNet V2: An effective deep network architecture for fast shot transition detectioncited by 166
- 2023SMPLer-X: Scaling Up Expressive Human Pose and Shape Estimationcited by 154
- 2022MOTRv2: Bootstrapping End-to-End Multi-Object Tracking by Pretrained Object Detectorscited by 148
- 2025LLaDA-V: Large Language Diffusion Models with Visual Instruction Tuningcited by 145
- 2022Hybrid Transformers for Music Source Separationcited by 134
- 2023MMM: Generative Masked Motion Modelcited by 129
- 2024World-Grounded Human Motion Recovery via Gravity-View Coordinatescited by 127
- 2024WavTokenizer: an Efficient Acoustic Discrete Codec Tokenizer for Audio Language Modelingcited by 124
- 2023EMAGE: Towards Unified Holistic Co-Speech Gesture Generation via Expressive Masked Audio Gesture Modelingcited by 101
- 2023TLControl: Trajectory and Language Control for Human Motion Synthesiscited by 93
- 2025MotionStreamer: Streaming Motion Generation via Diffusion-based Autoregressive Model in Causal Latent Spacecited by 82
- 2024OpenHumanVid: A Large-Scale High-Quality Dataset for Enhancing Human-Centric Video Generationcited by 77
- 2025RF-DETR: Neural Architecture Search for Real-Time Detection Transformerscited by 70
- 2025Go to Zero: Towards Zero-shot Motion Generation with Million-scale Datacited by 67
- 2024BAMM: Bidirectional Autoregressive Motion Modelcited by 54
- 2024ScaMo: Exploring the Scaling Law in Autoregressive Motion Generation Modelcited by 52
- 2024Generating Human Motion in 3D Scenes from Text Descriptionscited by 49
- 2024The Language of Motion: Unifying Verbal and Non-verbal Language of 3D Human Motioncited by 46
- 2025SnapMoGen: Human Motion Generation from Expressive Textscited by 44
- 2025GENMO: A GENeralist Model for Human MOtioncited by 43
- 2025Motion Anything: Any to Motion Generationcited by 40
- 2024MaskControl: Spatio-Temporal Control for Masked Motion Synthesiscited by 37
- 2026Kimodo: Scaling Controllable Human Motion Generationcited by 33
- 2025HY-Motion 1.0: Scaling Flow Matching Models for Text-To-Motion Generationcited by 32
- 2025Motion-X++: A Large-Scale Multimodal 3D Whole-body Human Motion Datasetcited by 30
50 most cited of 56 All 56 (JSON)
Built on by
Citing papers per year: 2026: 1.
1 paper.