# VoxelTrack: Multi-Person 3D Human Pose Estimation and Tracking in the Wild

arXiv: https://arxiv.org/abs/2108.02452 · HTML: https://mlsanity.com/paper/2108.02452/

## Key facts

- Cited by 39 papers in the map (data through 14 Oct 2026).
- Builds on 58 of its 96 references that are in the map.
- First posted on arXiv 5 Aug 2021.

## Builds on (most cited first, 30 of 58)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [MobileNetV2: Inverted Residuals and Linear Bottlenecks](https://mlsanity.com/paper/1801.04381/) (2018)
- [Deep High-Resolution Representation Learning for Human Pose Estimation](https://mlsanity.com/paper/1902.09212/) (2019)
- [Objects as Points](https://mlsanity.com/paper/1904.07850/) (2019)
- [Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields](https://mlsanity.com/paper/1611.08050/) (2016)
- [Stacked Hourglass Networks for Human Pose Estimation](https://mlsanity.com/paper/1603.06937/) (2016)
- [Simple Online and Realtime Tracking with a Deep Association Metric](https://mlsanity.com/paper/1703.07402/) (2017)
- [Simple Online and Realtime Tracking](https://mlsanity.com/paper/1602.00763/) (2016)
- [Performance Measures and a Data Set for Multi-Target, Multi-Camera Tracking](https://mlsanity.com/paper/1609.01775/) (2016)
- [Simple Baselines for Human Pose Estimation and Tracking](https://mlsanity.com/paper/1804.06208/) (2018)
- [Monocular 3D Human Pose Estimation In The Wild Using Improved CNN Supervision](https://mlsanity.com/paper/1611.09813/) (2016)
- [FairMOT: On the Fairness of Detection and Re-Identification in Multiple Object Tracking](https://mlsanity.com/paper/2004.01888/) (2020)
- [Beyond Part Models: Person Retrieval with Refined Part Pooling (and a Strong Convolutional Baseline)](https://mlsanity.com/paper/1711.09349/) (2017)
- [Deep Layer Aggregation](https://mlsanity.com/paper/1707.06484/) (2017)
- [A simple yet effective baseline for 3d human pose estimation](https://mlsanity.com/paper/1705.03098/) (2017)
- [Tracking without bells and whistles](https://mlsanity.com/paper/1903.05625/) (2019)
- [Cascaded Pyramid Network for Multi-Person Pose Estimation](https://mlsanity.com/paper/1711.07319/) (2017)
- [Towards Real-Time Multi-Object Tracking](https://mlsanity.com/paper/1909.12605/) (2019)
- [HigherHRNet: Scale-Aware Representation Learning for Bottom-Up Human Pose Estimation](https://mlsanity.com/paper/1908.10357/) (2019)
- [Associative Embedding: End-to-End Learning for Joint Detection and Grouping](https://mlsanity.com/paper/1611.05424/) (2016)
- [Panoptic Studio: A Massively Multiview System for Social Interaction Capture](https://mlsanity.com/paper/1612.03153/) (2016)
- [Monocular Total Capture: Posing Face, Body, and Hands in the Wild](https://mlsanity.com/paper/1812.01598/) (2018)
- [Learnable Triangulation of Human Pose](https://mlsanity.com/paper/1905.05754/) (2019)
- [Towards Accurate Multi-person Pose Estimation in the Wild](https://mlsanity.com/paper/1701.01779/) (2017)
- [Joint Training of a Convolutional Network and a Graphical Model for Human Pose Estimation](https://mlsanity.com/paper/1406.2984/) (2014)
- [PifPaf: Composite Fields for Human Pose Estimation](https://mlsanity.com/paper/1903.06593/) (2019)
- [DeepCut: Joint Subset Partition and Labeling for Multi Person Pose Estimation](https://mlsanity.com/paper/1511.06645/) (2015)
- [V2V-PoseNet: Voxel-to-Voxel Prediction Network for Accurate 3D Hand and Human Pose Estimation from a Single Depth Map](https://mlsanity.com/paper/1711.07399/) (2017)

## Built on by (most cited first, 30 of 39)

- [ByteTrack: Multi-Object Tracking by Associating Every Detection Box](https://mlsanity.com/paper/2110.06864/) (2021)
- [EgoHumans: An Egocentric 3D Multi-Human Benchmark](https://mlsanity.com/paper/2305.16487/) (2023)
- [3D Human Mesh Estimation from Virtual Markers](https://mlsanity.com/paper/2303.11726/) (2023)
- [Faster VoxelPose: Real-time 3D Human Pose Estimation by Orthographic Projection](https://mlsanity.com/paper/2207.10955/) (2022)
- [Scene-aware Egocentric 3D Human Pose Estimation](https://mlsanity.com/paper/2212.11684/) (2022)
- [MMPTRACK: Large-scale Densely Annotated Multi-camera Multiple People Tracking Benchmark](https://mlsanity.com/paper/2111.15157/) (2021)
- [HMD-Poser: On-Device Real-time Human Motion Tracking from Scalable Sparse Observations](https://mlsanity.com/paper/2403.03561/) (2024)
- [TEMPO: Efficient Multi-View Pose Estimation, Tracking, and Forecasting](https://mlsanity.com/paper/2309.07910/) (2023)
- [SelfPose3d: Self-Supervised Multi-Person Multi-View 3d Pose Estimation](https://mlsanity.com/paper/2404.02041/) (2024)
- [ByteTrackV2: 2D and 3D Multi-Object Tracking by Associating Every Detection Box](https://mlsanity.com/paper/2303.15334/) (2023)
- [Multiple View Geometry Transformers for 3D Human Pose Estimation](https://mlsanity.com/paper/2311.10983/) (2023)
- [WorldPose: A World Cup Dataset for Global 3D Human Pose Estimation](https://mlsanity.com/paper/2501.02771/) (2025)
- [Deep learning for 3D human pose estimation and mesh recovery: A survey](https://mlsanity.com/paper/2402.18844/) (2024)
- [VirtualPose: Learning Generalizable 3D Human Pose Models from Virtual Data](https://mlsanity.com/paper/2207.09949/) (2022)
- [Correlation Pyramid Network for 3D Single Object Tracking](https://mlsanity.com/paper/2305.09195/) (2023)
- [Markerless Multi-view 3D Human Pose Estimation: a survey](https://mlsanity.com/paper/2407.03817/) (2024)
- [Robust Multi-Object Tracking by Marginal Inference](https://mlsanity.com/paper/2208.03727/) (2022)
- [AvatarPose: Avatar-guided 3D Pose Estimation of Close Human Interaction from Sparse Multi-view Videos](https://mlsanity.com/paper/2408.02110/) (2024)
- [SimpleTrack: Rethinking and Improving the JDE Approach for Multi-Object Tracking](https://mlsanity.com/paper/2203.03985/) (2022)
- [Snipper: A Spatiotemporal Transformer for Simultaneous Multi-Person 3D Pose Estimation Tracking and Forecasting on a Video Snippet](https://mlsanity.com/paper/2207.04320/) (2022)
- [Human Pose-based Estimation, Tracking and Action Recognition with Deep Learning: A Survey](https://mlsanity.com/paper/2310.13039/) (2023)
- [SportsTrack: An Innovative Method for Tracking Athletes in Sports Scenes](https://mlsanity.com/paper/2211.07173/) (2022)
- [A Unified Multi-view Multi-person Tracking Framework](https://mlsanity.com/paper/2302.03820/) (2023)
- [MITracker: Multi-View Integration for Visual Object Tracking](https://mlsanity.com/paper/2502.20111/) (2025)
- [A survey of top-down approaches for human pose estimation](https://mlsanity.com/paper/2202.02656/) (2022)
- [VTP: Volumetric Transformer for Multi-view Multi-person 3D Pose Estimation](https://mlsanity.com/paper/2205.12602/) (2022)
- [Bottom-up approaches for multi-person pose estimation and it's applications: A brief review](https://mlsanity.com/paper/2112.11834/) (2021)
- [Multi-View Person Matching and 3D Pose Estimation with Arbitrary Uncalibrated Camera Networks](https://mlsanity.com/paper/2312.01561/) (2023)
- [Sports Analysis and VR Viewing System Based on Player Tracking and Pose Estimation with Multimodal and Multiview Sensors](https://mlsanity.com/paper/2405.01112/) (2024)
- [Leveraging Systematic Knowledge of 2D Transformations](https://mlsanity.com/paper/2206.00893/) (2022)

## Abstract (arXiv)

We present VoxelTrack for multi-person 3D pose estimation and tracking from a few cameras which are separated by wide baselines. It employs a multi-branch network to jointly estimate 3D poses and re-identification (Re-ID) features for all people in the environment. In contrast to previous efforts which require to establish cross-view correspondence based on noisy 2D pose estimates, it directly estimates and tracks 3D poses from a 3D voxel-based representation constructed from multi-view images. We first discretize the 3D space by regular voxels and compute a feature vector for each voxel by averaging the body joint heatmaps that are inversely projected from all views. We estimate 3D poses from the voxel representation by predicting whether each voxel contains a particular body joint. Similarly, a Re-ID feature is computed for each voxel which is used to track the estimated 3D poses over time. The main advantage of the approach is that it avoids making any hard decisions based on individual images. The approach can robustly estimate and track 3D poses even when people are severely occluded in some cameras. It outperforms the state-of-the-art methods by a large margin on three public datasets including Shelf, Campus and CMU Panoptic.