# The MI-Motion Dataset and Benchmark for 3D Multi-Person Motion Prediction

arXiv: https://arxiv.org/abs/2306.13566 · HTML: https://mlsanity.com/paper/2306.13566/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 39 of its 63 references that are in the map.
- First posted on arXiv 23 Jun 2023.

## Builds on (most cited first, 30 of 39)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling](https://mlsanity.com/paper/1803.01271/) (2018)
- [AMASS: Archive of Motion Capture as Surface Shapes](https://mlsanity.com/paper/1904.03278/) (2019)
- [Generating Sequences With Recurrent Neural Networks](https://mlsanity.com/paper/1308.0850/) (2013)
- [Convolutional Sequence to Sequence Learning](https://mlsanity.com/paper/1705.03122/) (2017)
- [Temporal Convolutional Networks for Action Segmentation and Detection](https://mlsanity.com/paper/1611.05267/) (2016)
- [AI Choreographer: Music Conditioned 3D Dance Generation with AIST++](https://mlsanity.com/paper/2101.08779/) (2021)
- [On human motion prediction using recurrent neural networks](https://mlsanity.com/paper/1705.02445/) (2017)
- [Social-STGCNN: A Social Spatio-Temporal Graph Convolutional Neural Network for Human Trajectory Prediction](https://mlsanity.com/paper/2002.11927/) (2020)
- [Learning Trajectory Dependencies for Human Motion Prediction](https://mlsanity.com/paper/1908.05436/) (2019)
- [Fast and Furious: Real Time End-to-End 3D Detection, Tracking and Motion Forecasting with a Single Convolutional Net](https://mlsanity.com/paper/2012.12395/) (2020)
- [Single-Shot Multi-Person 3D Pose Estimation From Monocular RGB](https://mlsanity.com/paper/1712.03453/) (2017)
- [Long-term Human Motion Prediction with Scene Context](https://mlsanity.com/paper/2007.03672/) (2020)
- [HP-GAN: Probabilistic 3D human motion prediction via GAN](https://mlsanity.com/paper/1711.09561/) (2017)
- [History Repeats Itself: Human Motion Prediction via Motion Attention](https://mlsanity.com/paper/2007.11755/) (2020)
- [A Spatio-temporal Transformer for 3D Human Motion Prediction](https://mlsanity.com/paper/2004.08692/) (2020)
- [Dynamic Multiscale Graph Neural Networks for 3D Skeleton-Based Human Motion Prediction](https://mlsanity.com/paper/2003.08802/) (2020)
- [Human Motion Prediction via Spatio-Temporal Inpainting](https://mlsanity.com/paper/1812.05478/) (2018)
- [MSR-GCN: Multi-Scale Residual Graph Convolution Networks for Human Motion Prediction](https://mlsanity.com/paper/2108.07152/) (2021)
- [Back to MLP: A Simple Baseline for Human Motion Prediction](https://mlsanity.com/paper/2207.01567/) (2022)
- [Space-Time-Separable Graph Convolutional Network for Pose Forecasting](https://mlsanity.com/paper/2110.04573/) (2021)
- [You2Me: Inferring Body Pose in Egocentric Video via First and Second Person Interactions](https://mlsanity.com/paper/1904.09882/) (2019)
- [GIMO: Gaze-Informed Human Motion Prediction in Context](https://mlsanity.com/paper/2204.09443/) (2022)
- [Multi-Person Extreme Motion Prediction](https://mlsanity.com/paper/2105.08825/) (2021)
- [Progressively Generating Better Initial Guesses Towards Next Stages for High-Quality Human Motion Prediction](https://mlsanity.com/paper/2203.16051/) (2022)
- [Multi-Person 3D Motion Prediction with Multi-Range Transformers](https://mlsanity.com/paper/2111.12073/) (2021)
- [BiHMP-GAN: Bidirectional 3D Human Motion Prediction GAN](https://mlsanity.com/paper/1812.02591/) (2018)
- [TRiPOD: Human Trajectory and Pose Dynamics Forecasting in the Wild](https://mlsanity.com/paper/2104.04029/) (2021)
- [Socially and Contextually Aware Human Motion and Pose Forecasting](https://mlsanity.com/paper/2007.06843/) (2020)
- [Spatio-Temporal Gating-Adjacency GCN for Human Motion Prediction](https://mlsanity.com/paper/2203.01474/) (2022)

## Built on by (most cited first, 3 of 3)

- [Robot Interaction Behavior Generation based on Social Motion Forecasting for Human-Robot Interaction](https://mlsanity.com/paper/2402.04768/) (2024)
- [Discovering Symmetry Groups with Flow Matching](https://mlsanity.com/paper/2512.20043/) (2025)
- [CoMPAS3D: A Dataset and Benchmark for Interactive Motion](https://mlsanity.com/paper/2507.19684/) (2025)

## Abstract (arXiv)

3D multi-person motion prediction is a challenging task that involves modeling individual behaviors and interactions between people. Despite the emergence of approaches for this task, comparing them is difficult due to the lack of standardized training settings and benchmark datasets. In this paper, we introduce the Multi-Person Interaction Motion (MI-Motion) Dataset, which includes skeleton sequences of multiple individuals collected by motion capture systems and refined and synthesized using a game engine. The dataset contains 167k frames of interacting people's skeleton poses and is categorized into 5 different activity scenes. To facilitate research in multi-person motion prediction, we also provide benchmarks to evaluate the performance of prediction methods in three settings: short-term, long-term, and ultra-long-term prediction. Additionally, we introduce a novel baseline approach that leverages graph and temporal convolutional networks, which has demonstrated competitive results in multi-person motion prediction. We believe that the proposed MI-Motion benchmark dataset and baseline will facilitate future research in this area, ultimately leading to better understanding and modeling of multi-person interactions.