Grounding Intelligence in Movement
Not on the map. Why?
Abstract (arXiv)
Recent advances in machine learning have dramatically improved our ability to model language, vision, and other high-dimensional data, yet they continue to struggle with one of the most fundamental aspects of biological systems: movement. Across neuroscience, medicine, robotics, and ethology, movement is essential for interpreting behavior, predicting intent, and enabling interaction. Despite its core significance in our intelligence, movement is often treated as an afterthought rather than as a rich and structured modality in its own right. This reflects a deeper fragmentation in how movement data is collected and modeled, often constrained by task-specific goals and domain-specific assumptions. But movement is not domain-bound. It reflects shared physical constraints, conserved morphological structures, and purposeful dynamics that cut across species and settings. We argue that movement should be treated as a primary modeling target for AI. It is inherently structured and grounded in embodiment and physics. This structure, often allowing for compact, lower-dimensional representations (e.g., pose), makes it more interpretable and computationally tractable to model than raw, high-dimensional sensory inputs. Developing models that can learn from and generalize across diverse movement data will not only advance core capabilities in generative modeling and control, but also create a shared foundation for understanding behavior across biological and artificial systems. Movement is not just an outcome, it is a window into how intelligent systems engage with the world.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 74 of its 106 references that are in the map. First posted on arXiv 3 Jul 2025.
Builds on
74 of 106 references.
References · 74
- 2020Denoising Diffusion Probabilistic Modelscited by 20,851
- 2023GPT-4 Technical Reportcited by 18,132
- 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
- 2018World Modelscited by 1,444
- 2023Video-LLaVA: Learning United Visual Representation by Alignment Before Projectioncited by 1,385
- 2019AMASS: Archive of Motion Capture as Surface Shapescited by 1,208
- 2023Mastering Diverse Domains through World Modelscited by 1,193
- 2023ImageBind: One Embedding Space To Bind Them Allcited by 1,162
- 2022Human Motion Diffusion Modelcited by 1,062
- 2024WorldSimBench: Towards Video Generation Models as World Simulatorscited by 867
- 2025Cosmos World Foundation Model Platform for Physical AIcited by 833
- 2022A Generalist Agentcited by 802
38 more
- 2022MotionDiffuse: Text-Driven Human Motion Generation with Diffusion Modelcited by 768
- 2023DETRs Beat YOLOs on Real-time Object Detectioncited by 526
- 2023MotionGPT: Human Motion as a Foreign Languagecited by 521
- 2023VideoMAE V2: Scaling Video Masked Autoencoders with Dual Maskingcited by 516
- 2024MOMENT: A Family of Open Time-series Foundation Modelscited by 453
- 2022ViTPose: Simple Vision Transformer Baselines for Human Pose Estimationcited by 450
- 2022Unified-IO: A Unified Model for Vision, Language, and Multi-Modal Taskscited by 420
- 2022PhysDiff: Physics-Guided Human Motion Diffusion Modelcited by 365
- 2023AudioPaLM: A Large Language Model That Can Speak and Listencited by 333
- 20163D Menagerie: Modeling the 3D shape and pose of animalscited by 310
- 2019A Short Note on the Kinetics-700 Human Action Datasetcited by 308
- 2024Foundation Models for Time Series Analysis: A Tutorial and Surveycited by 234
- 2019Human Motion Trajectory Prediction: A Surveycited by 231
- 2023Motion-X: A Large-scale 3D Expressive Whole-body Human Motion Datasetcited by 220
- 2018Markerless tracking of user-defined features with deep learningcited by 203
- 2023Project Aria: A New Tool for Egocentric Multi-Modal AI Researchcited by 201
- 2023SMPLer-X: Scaling Up Expressive Human Pose and Shape Estimationcited by 154
- 2021EgoBody: Human Body Shape and Motion of Interacting People from Head-Mounted Devicescited by 136
- 2023Meta-Transformer: A Unified Framework for Multimodal Learningcited by 136
- 2024iVideoGPT: Interactive VideoGPTs are Scalable World Modelscited by 136
- 2023On Memorization in Diffusion Modelscited by 122
- 2024Nymeria: A Massive Collection of Multimodal Egocentric Daily Motion in the Wildcited by 109
- 2024Evaluating the World Model Implicit in a Generative Modelcited by 107
- 2023DiffPoseTalk: Speech-Driven Stylistic 3D Facial Animation and Head Pose Generation via Diffusion Modelscited by 81
- 2022MyoSuite -- A contact-rich simulation suite for musculoskeletal motor controlcited by 80
- 2023ChatPose: Chatting about 3D Human Posecited by 71
- 2025PromptHMR: Promptable Human Mesh Recoverycited by 70
- 2023Neuro-GPT: Towards A Foundation Model for EEGcited by 69
- 2022ViTPose++: Vision Transformer for Generic Body Pose Estimationcited by 65
- 2023A Unified, Scalable Framework for Neural Population Decodingcited by 59
- 2020MoVi: A Large Multipurpose Motion and Video Datasetcited by 58
- 2025HumanOmni: A Large Vision-Speech Language Model for Human-Centric Video Understandingcited by 58
- 2024AnyGraph: Graph Foundation Model in the Wildcited by 51
- 2024OmniJet-α: The first cross-task foundation model for particle physicscited by 41
- 20203D Bird Reconstruction: a Dataset, Model, and Shape Recovery from a Single Viewcited by 40
- 2022SuperAnimal pretrained pose estimation models for behavioral analysiscited by 38
- 2024OmniBind: Large-scale Omni Multimodal Representation via Binding Spacescited by 38
- 2023Robot Learning in the Era of Foundation Models: A Surveycited by 37
50 most cited of 74 All 74 (JSON)
Built on by
Citing papers per year: 2026: 1.
1 paper.