# MoVi: A Large Multipurpose Motion and Video Dataset

arXiv: https://arxiv.org/abs/2003.01888 · HTML: https://mlsanity.com/paper/2003.01888/

## Key facts

- Cited by 58 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 Mar 2020.

## Built on by (most cited first, 30 of 58)

- [BABEL: Bodies, Action and Behavior with English Labels](https://mlsanity.com/paper/2106.09696/) (2021)
- [Nymeria: A Massive Collection of Multimodal Egocentric Daily Motion in the Wild](https://mlsanity.com/paper/2406.09905/) (2024)
- [Neural-GIF: Neural Generalized Implicit Functions for Animating People in Clothing](https://mlsanity.com/paper/2108.08807/) (2021)
- [MM-Fi: Multi-Modal Non-Intrusive 4D Human Dataset for Versatile Wireless Sensing](https://mlsanity.com/paper/2305.10345/) (2023)
- [The Wanderings of Odysseus in 3D Scenes](https://mlsanity.com/paper/2112.09251/) (2021)
- [Neural Localizer Fields for Continuous 3D Human Pose and Shape Estimation](https://mlsanity.com/paper/2407.07532/) (2024)
- [UnrealEgo: A New Dataset for Robust Egocentric 3D Human Motion Capture](https://mlsanity.com/paper/2208.01633/) (2022)
- [Learning 3D Human Pose Estimation from Dozens of Datasets using a Geometry-Aware Autoencoder to Bridge Between Skeleton Formats](https://mlsanity.com/paper/2212.14474/) (2022)
- [mRI: Multi-modal 3D Human Pose Estimation Dataset using mmWave, RGB-D, and Inertial Sensors](https://mlsanity.com/paper/2210.08394/) (2022)
- [Self-Correcting Self-Consuming Loops for Generative Model Training](https://mlsanity.com/paper/2402.07087/) (2024)
- [SOMA: Solving Optical Marker-Based MoCap Automatically](https://mlsanity.com/paper/2110.04431/) (2021)
- [BEDLAM2.0: Synthetic Humans and Cameras in Motion](https://mlsanity.com/paper/2511.14394/) (2025)
- [Differentiable Biomechanics Unlocks Opportunities for Markerless Motion Capture](https://mlsanity.com/paper/2402.17192/) (2024)
- [BioPose: Biomechanically-accurate 3D Pose Estimation from Monocular Videos](https://mlsanity.com/paper/2501.07800/) (2025)
- [Story-to-Motion: Synthesizing Infinite and Controllable Character Animation from Long Text](https://mlsanity.com/paper/2311.07446/) (2023)
- [From Sparse Signal to Smooth Motion: Real-Time Motion Generation with Rolling Prediction Models](https://mlsanity.com/paper/2504.05265/) (2025)
- [RL from Physical Feedback: Aligning Large Motion Models with Humanoid Control](https://mlsanity.com/paper/2506.12769/) (2025)
- [Markerless Motion Capture and Biomechanical Analysis Pipeline](https://mlsanity.com/paper/2303.10654/) (2023)
- [Towards Single Camera Human 3D-Kinematics](https://mlsanity.com/paper/2301.05435/) (2023)
- [Differentiable Biomechanics for Markerless Motion Capture in Upper Limb Stroke Rehabilitation: A Comparison with Optical Motion Capture](https://mlsanity.com/paper/2411.14992/) (2024)
- [Move-in-2D: 2D-Conditioned Human Motion Generation](https://mlsanity.com/paper/2412.13185/) (2024)
- [Portable Biomechanics Laboratory: Clinically Accessible Movement Analysis from a Handheld Smartphone](https://mlsanity.com/paper/2507.08268/) (2025)
- [OpenCapBench: A Benchmark to Bridge Pose Estimation and Biomechanics](https://mlsanity.com/paper/2406.09788/) (2024)
- [Fusing uncalibrated IMUs and handheld smartphone video to reconstruct knee kinematics](https://mlsanity.com/paper/2405.17368/) (2024)
- [Self-Supervised Learning of Gait-Based Biomarkers](https://mlsanity.com/paper/2307.16321/) (2023)
- [KinTwin: Imitation Learning with Torque and Muscle Driven Biomechanical Models Enables Precise Replication of Able-Bodied and Impaired Movement from Markerless Motion Capture](https://mlsanity.com/paper/2505.13436/) (2025)
- [Monocular Biomechanical Tracking of Fingers with Inverse Kinematics to Foundation Models](https://mlsanity.com/paper/2605.09258/) (2026)
- [Calibrated Uncertainty for Trustworthy Clinical Gait Analysis Using Probabilistic Multiview Markerless Motion Capture](https://mlsanity.com/paper/2601.22412/) (2026)
- [AIMusicGuru: Music Assisted Human Pose Correction](https://mlsanity.com/paper/2203.12829/) (2022)
- [BLiSS: Bootstrapped Linear Shape Space](https://mlsanity.com/paper/2309.01765/) (2023)

## Abstract (arXiv)

Human movements are both an area of intense study and the basis of many applications such as character animation. For many applications, it is crucial to identify movements from videos or analyze datasets of movements. Here we introduce a new human Motion and Video dataset MoVi, which we make available publicly. It contains 60 female and 30 male actors performing a collection of 20 predefined everyday actions and sports movements, and one self-chosen movement. In five capture rounds, the same actors and movements were recorded using different hardware systems, including an optical motion capture system, video cameras, and inertial measurement units (IMU). For some of the capture rounds, the actors were recorded when wearing natural clothing, for the other rounds they wore minimal clothing. In total, our dataset contains 9 hours of motion capture data, 17 hours of video data from 4 different points of view (including one hand-held camera), and 6.6 hours of IMU data. In this paper, we describe how the dataset was collected and post-processed; We present state-of-the-art estimates of skeletal motions and full-body shape deformations associated with skeletal motion. We discuss examples for potential studies this dataset could enable.