# 4D Association Graph for Realtime Multi-person Motion Capture Using Multiple Video Cameras

arXiv: https://arxiv.org/abs/2002.12625 · HTML: https://mlsanity.com/paper/2002.12625/

## Key facts

- Cited by 49 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Feb 2020.

## Built on by (most cited first, 30 of 49)

- [Capturing and Inferring Dense Full-Body Human-Scene Contact](https://mlsanity.com/paper/2206.09553/) (2022)
- [GLAMR: Global Occlusion-Aware Human Mesh Recovery with Dynamic Cameras](https://mlsanity.com/paper/2112.01524/) (2021)
- [EgoBody: Human Body Shape and Motion of Interacting People from Head-Mounted Devices](https://mlsanity.com/paper/2112.07642/) (2021)
- [SPEC: Seeing People in the Wild with an Estimated Camera](https://mlsanity.com/paper/2110.00620/) (2021)
- [Learning Motion Priors for 4D Human Body Capture in 3D Scenes](https://mlsanity.com/paper/2108.10399/) (2021)
- [InterCap: Joint Markerless 3D Tracking of Humans and Objects in Interaction](https://mlsanity.com/paper/2209.12354/) (2022)
- [DeepMultiCap: Performance Capture of Multiple Characters Using Sparse Multiview Cameras](https://mlsanity.com/paper/2105.00261/) (2021)
- [HOI-M3:Capture Multiple Humans and Objects Interaction within Contextual Environment](https://mlsanity.com/paper/2404.00299/) (2024)
- [EgoLocate: Real-time Motion Capture, Localization, and Mapping with Sparse Body-mounted Sensors](https://mlsanity.com/paper/2305.01599/) (2023)
- [Faster VoxelPose: Real-time 3D Human Pose Estimation by Orthographic Projection](https://mlsanity.com/paper/2207.10955/) (2022)
- [Graph-Based 3D Multi-Person Pose Estimation Using Multi-View Images](https://mlsanity.com/paper/2109.05885/) (2021)
- [PACE: Human and Camera Motion Estimation from in-the-wild Videos](https://mlsanity.com/paper/2310.13768/) (2023)
- [Lightweight Multi-person Total Motion Capture Using Sparse Multi-view Cameras](https://mlsanity.com/paper/2108.10378/) (2021)
- [WHAC: World-grounded Humans and Cameras](https://mlsanity.com/paper/2403.12959/) (2024)
- [Towards Open Domain Text-Driven Synthesis of Multi-Person Motions](https://mlsanity.com/paper/2405.18483/) (2024)
- [Dynamic Multi-Person Mesh Recovery From Uncalibrated Multi-View Cameras](https://mlsanity.com/paper/2110.10355/) (2021)
- [Shape-aware Multi-Person Pose Estimation from Multi-View Images](https://mlsanity.com/paper/2110.02330/) (2021)
- [Tracking People with 3D Representations](https://mlsanity.com/paper/2111.07868/) (2021)
- [TEMPO: Efficient Multi-View Pose Estimation, Tracking, and Forecasting](https://mlsanity.com/paper/2309.07910/) (2023)
- [Multiple View Geometry Transformers for 3D Human Pose Estimation](https://mlsanity.com/paper/2311.10983/) (2023)
- [Tracking People by Predicting 3D Appearance, Location & Pose](https://mlsanity.com/paper/2112.04477/) (2021)
- [WorldPose: A World Cup Dataset for Global 3D Human Pose Estimation](https://mlsanity.com/paper/2501.02771/) (2025)
- [SocialGen: Modeling Multi-Human Social Interaction with Language Models](https://mlsanity.com/paper/2503.22906/) (2025)
- [Part-Aware Measurement for Robust Multi-View Multi-Human 3D Pose Estimation and Tracking](https://mlsanity.com/paper/2106.11589/) (2021)
- [RAM: Recover Any 3D Human Motion in-the-Wild](https://mlsanity.com/paper/2603.19929/) (2026)
- [Delving Deep into Pixel Alignment Feature for Accurate Multi-view Human Mesh Recovery](https://mlsanity.com/paper/2301.06020/) (2023)
- [ProxyCap: Real-time Monocular Full-body Capture in World Space via Human-Centric Proxy-to-Motion Learning](https://mlsanity.com/paper/2307.01200/) (2023)
- [MV-SSM: Multi-View State Space Modeling for 3D Human Pose Estimation](https://mlsanity.com/paper/2509.00649/) (2025)
- [A Dual-Masked Auto-Encoder for Robust Motion Capture with Spatial-Temporal Skeletal Token Completion](https://mlsanity.com/paper/2207.07381/) (2022)
- [Reconstructing Close Human Interactions from Multiple Views](https://mlsanity.com/paper/2401.16173/) (2024)

## Abstract (arXiv)

This paper contributes a novel realtime multi-person motion capture algorithm using multiview video inputs. Due to the heavy occlusions in each view, joint optimization on the multiview images and multiple temporal frames is indispensable, which brings up the essential challenge of realtime efficiency. To this end, for the first time, we unify per-view parsing, cross-view matching, and temporal tracking into a single optimization framework, i.e., a 4D association graph that each dimension (image space, viewpoint and time) can be treated equally and simultaneously. To solve the 4D association graph efficiently, we further contribute the idea of 4D limb bundle parsing based on heuristic searching, followed with limb bundle assembling by proposing a bundle Kruskal's algorithm. Our method enables a realtime online motion capture system running at 30fps using 5 cameras on a 5-person scene. Benefiting from the unified parsing, matching and tracking constraints, our method is robust to noisy detection, and achieves high-quality online pose reconstruction quality. The proposed method outperforms the state-of-the-art method quantitatively without using high-level appearance information. We also contribute a multiview video dataset synchronized with a marker-based motion capture system for scientific evaluation.