mlsanityData to 14 Oct 2026

MoFM: A Large-Scale Human Motion Foundation Model

arXiv 2502.05432 ↗First posted 8 Feb 2025cs.CV cs.LGcited by 5

Not on the map. Why?

Abstract (arXiv)

Foundation Models (FM) have increasingly drawn the attention of researchers due to their scalability and generalization across diverse tasks. Inspired by the success of FMs and the principles that have driven advancements in Large Language Models (LLMs), we introduce MoFM as a novel Motion Foundation Model. MoFM is designed for the semantic understanding of complex human motions in both time and space. To facilitate large-scale training, MotionBook, a comprehensive human motion dictionary of discretized motions is designed and employed. MotionBook utilizes Thermal Cubes to capture spatio-temporal motion heatmaps, applying principles from discrete variational models to encode human movements into discrete units for a more efficient and scalable representation. MoFM, trained on a large corpus of motion data, provides a foundational backbone adaptable to diverse downstream tasks, supporting paradigms such as one-shot, unsupervised, and supervised tasks. This versatility makes MoFM well-suited for a wide range of motion-based applications.

Key facts. Cited by 5 papers in the map (data through 14 Oct 2026). Builds on 79 of its 104 references that are in the map. First posted on arXiv 8 Feb 2025.

Builds on

79 of 104 references.

References · 79

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2020Language Models are Few-Shot Learnerscited by 32,419
  4. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  5. 2023Segment Anythingcited by 8,234
  6. 2017Neural Discrete Representation Learningcited by 4,712
  7. 2021Zero-Shot Text-to-Image Generationcited by 4,288
  8. 2020Supervised Contrastive Learningcited by 2,829
  9. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,565
  10. 2021BEiT: BERT Pre-Training of Image Transformerscited by 2,375
  11. 2021Is Space-Time Attention All You Need for Video Understanding?cited by 1,620
  12. 2019Generating Diverse High-Fidelity Images with VQ-VAE-2cited by 1,387
38 more
  1. 2019Deep High-Resolution Representation Learning for Human Pose Estimationcited by 1,339
  2. 2018Spatial Temporal Graph Convolutional Networks for Skeleton-Based Action Recognitioncited by 1,209
  3. 2016NTU RGB+D: A Large Scale Dataset for 3D Human Activity Analysiscited by 686
  4. 2019NTU RGB+D 120: A Large-Scale Benchmark for 3D Human Activity Understandingcited by 533
  5. 2022InternImage: Exploring Large-Scale Vision Foundation Models with Deformable Convolutionscited by 482
  6. 2017Future Frame Prediction for Anomaly Detection -- A New Baselinecited by 391
  7. 2020Disentangling and Unifying Graph Convolutions for Skeleton-Based Action Recognitioncited by 343
  8. 20213D Human Pose Estimation with Spatial and Temporal Transformerscited by 272
  9. 2021Mesh Graphormercited by 265
  10. 2021Revisiting Skeleton-based Action Recognitioncited by 257
  11. 2016Learning Temporal Regularity in Video Sequencescited by 254
  12. 2019HigherHRNet: Scale-Aware Representation Learning for Bottom-Up Human Pose Estimationcited by 247
  13. 2019Actional-Structural Graph Convolutional Networks for Skeleton-based Action Recognitioncited by 228
  14. 2018CrowdPose: Efficient Crowded Scenes Pose Estimation and A New Benchmarkcited by 214
  15. 2022AlphaPose: Whole-Body Regional Multi-Person Pose Estimation and Tracking in Real-Timecited by 200
  16. 2022CREPE: Can Vision-Language Foundation Models Reason Compositionally?cited by 181
  17. 2022Fine-tuned CLIP Models are Efficient Video Learnerscited by 179
  18. 2022MixSTE: Seq2seq Mixed Spatio-Temporal Encoder for 3D Human Pose Estimation in Videocited by 159
  19. 2018Latent Space Autoregression for Novelty Detectioncited by 149
  20. 2016Spatio-Temporal LSTM with Trust Gates for 3D Human Action Recognitioncited by 143
  21. 2018Independently Recurrent Neural Network (IndRNN): Building A Longer and Deeper RNNcited by 143
  22. 2017Towards 3D Human Pose Estimation in the Wild: a Weakly-supervised Approachcited by 115
  23. 2023SAM-CLIP: Merging Vision Foundation Models towards Semantic and Spatial Understandingcited by 111
  24. 2018Ordinal Depth Supervision for 3D Human Pose Estimationcited by 107
  25. 20213D Human Action Representation Learning via Cross-View Consistency Pursuitcited by 106
  26. 2017View Adaptive Recurrent Neural Networks for High Performance Human Action Recognition from Skeleton Datacited by 105
  27. 2019Learning Regularity in Skeleton Trajectories for Anomaly Detection in Videoscited by 104
  28. 2022Generative Cooperative Learning for Unsupervised Video Anomaly Detectioncited by 102
  29. 2019Self-Supervised Learning of 3D Human Pose using Multi-view Geometrycited by 98
  30. 2019Graph Embedded Pose Clustering for Anomaly Detectioncited by 95
  31. 2021Graph Stacked Hourglass Networks for 3D Human Pose Estimationcited by 95
  32. 2017Interpretable 3D Human Action Analysis with Temporal Convolutional Networkscited by 89
  33. 2020A Background-Agnostic Framework with Adversarial Training for Abnormal Event Detection in Videocited by 86
  34. 2021Probabilistic Monocular 3D Human Pose Estimation with Normalizing Flowscited by 85
  35. 2022Multi-Scale Spatial Temporal Graph Convolutional Network for Skeleton-Based Action Recognitioncited by 71
  36. 2020Rethinking the Heatmap Regression for Bottom-up Human Pose Estimationcited by 69
  37. 2023K2: A Foundation Language Model for Geoscience Knowledge Understanding and Utilizationcited by 69
  38. 2022Video Anomaly Detection by Solving Decoupled Spatio-Temporal Jigsaw Puzzlescited by 62

50 most cited of 79 All 79 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2025: 3; 2026: 2.

5 papers.

  1. 2025MoCLIP: Motion-Aware Fine-Tuning and Distillation of CLIP for Human Motion Generationcited by 2
  2. 2025Adversarially-Refined VQ-GAN with Dense Motion Tokenization for Spatio-Temporal Heatmapscited by 2
  3. 2026DriveMotion: A Large-Scale Multi-Source Benchmark for Driver Motion Sequence Modeling and Forecastingcited by 1
  4. 2025Cognitive State Inference from VR Motion via Motion Foundation Modelcited by 0
  5. 2026World Motion Models: Flexible Sequence Modeling of SE(3) Trajectoriescited by 0