# Large Scale Real-World Multi-Person Tracking

arXiv: https://arxiv.org/abs/2211.02175 · HTML: https://mlsanity.com/paper/2211.02175/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- Builds on 51 of its 68 references that are in the map.
- First posted on arXiv 3 Nov 2022.

## Builds on (most cited first, 30 of 51)

- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019)
- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [Scalability in Perception for Autonomous Driving: Waymo Open Dataset](https://mlsanity.com/paper/1912.04838/) (2019)
- [FCOS: Fully Convolutional One-Stage Object Detection](https://mlsanity.com/paper/1904.01355/) (2019)
- [LSUN: Construction of a Large-scale Image Dataset using Deep Learning with Humans in the Loop](https://mlsanity.com/paper/1506.03365/) (2015)
- [BDD100K: A Diverse Driving Dataset for Heterogeneous Multitask Learning](https://mlsanity.com/paper/1805.04687/) (2018)
- [Objects as Points](https://mlsanity.com/paper/1904.07850/) (2019)
- [YOLOX: Exceeding YOLO Series in 2021](https://mlsanity.com/paper/2107.08430/) (2021)
- [Simple Online and Realtime Tracking with a Deep Association Metric](https://mlsanity.com/paper/1703.07402/) (2017)
- [Simple Online and Realtime Tracking](https://mlsanity.com/paper/1602.00763/) (2016)
- [ByteTrack: Multi-Object Tracking by Associating Every Detection Box](https://mlsanity.com/paper/2110.06864/) (2021)
- [Argoverse: 3D Tracking and Forecasting with Rich Maps](https://mlsanity.com/paper/1911.02620/) (2019)
- [Performance Measures and a Data Set for Multi-Target, Multi-Camera Tracking](https://mlsanity.com/paper/1609.01775/) (2016)
- [Fully-Convolutional Siamese Networks for Object Tracking](https://mlsanity.com/paper/1606.09549/) (2016)
- [MOT16: A Benchmark for Multi-Object Tracking](https://mlsanity.com/paper/1603.00831/) (2016)
- [TrackFormer: Multi-Object Tracking with Transformers](https://mlsanity.com/paper/2101.02702/) (2021)
- [FairMOT: On the Fairness of Detection and Re-Identification in Multiple Object Tracking](https://mlsanity.com/paper/2004.01888/) (2020)
- [Tracking Objects as Points](https://mlsanity.com/paper/2004.01177/) (2020)
- [SiamRPN++: Evolution of Siamese Visual Tracking with Very Deep Networks](https://mlsanity.com/paper/1812.11703/) (2018)
- [Deep Layer Aggregation](https://mlsanity.com/paper/1707.06484/) (2017)
- [MOT20: A benchmark for multi object tracking in crowded scenes](https://mlsanity.com/paper/2003.09003/) (2020)
- [Towards Real-Time Multi-Object Tracking](https://mlsanity.com/paper/1909.12605/) (2019)
- [A Short Note on the Kinetics-700 Human Action Dataset](https://mlsanity.com/paper/1907.06987/) (2019)
- [CrowdHuman: A Benchmark for Detecting Human in a Crowd](https://mlsanity.com/paper/1805.00123/) (2018)
- [One Thousand and One Hours: Self-driving Motion Prediction Dataset](https://mlsanity.com/paper/2006.14480/) (2020)
- [Quasi-Dense Similarity Learning for Multiple Object Tracking](https://mlsanity.com/paper/2006.06664/) (2020)
- [SiamCAR: Siamese Fully Convolutional Classification and Regression for Visual Tracking](https://mlsanity.com/paper/1911.07241/) (2019)

## Built on by (most cited first, 14 of 14)

- [Hybrid-SORT: Weak Cues Matter for Online Multi-Object Tracking](https://mlsanity.com/paper/2308.00783/) (2023)
- [Self-Supervised Multi-Object Tracking with Path Consistency](https://mlsanity.com/paper/2404.05136/) (2024)
- [DeconfuseTrack:Dealing with Confusion for Multi-Object Tracking](https://mlsanity.com/paper/2403.02767/) (2024)
- [Ponimator: Unfolding Interactive Pose for Versatile Human-human Interaction Animation](https://mlsanity.com/paper/2510.14976/) (2025)
- [Is Multiple Object Tracking a Matter of Specialization?](https://mlsanity.com/paper/2411.00553/) (2024)
- [Boundless: Generating Photorealistic Synthetic Data for Object Detection in Urban Streetscapes](https://mlsanity.com/paper/2409.03022/) (2024)
- [HERBench: A Benchmark for Multi-Evidence Integration in Video Question Answering](https://mlsanity.com/paper/2512.14870/) (2025)
- [Instance-level Visual Active Tracking with Occlusion-Aware Planning](https://mlsanity.com/paper/2604.21453/) (2026)
- [Constellation Dataset: Benchmarking High-Altitude Object Detection for an Urban Intersection](https://mlsanity.com/paper/2404.16944/) (2024)
- [SkeleTR: Towrads Skeleton-based Action Recognition in the Wild](https://mlsanity.com/paper/2309.11445/) (2023)
- [CARPE-ID: Continuously Adaptable Re-identification for Personalized Robot Assistance](https://mlsanity.com/paper/2310.19413/) (2023)
- [LITE: A Paradigm Shift in Multi-Object Tracking with Efficient ReID Feature Integration](https://mlsanity.com/paper/2409.04187/) (2024)
- [Learning-Augmented Scalable Linear Assignment Problem Optimization via Neural Dual Warm-Starts](https://mlsanity.com/paper/2605.09382/) (2026)
- [SAMOFT: Robust Multi-Object Tracking via Region and Flow](https://mlsanity.com/paper/2605.09417/) (2026)

## Abstract (arXiv)

This paper presents a new large scale multi-person tracking dataset -- textttPersonPath22, which is over an order of magnitude larger than currently available high quality multi-object tracking datasets such as MOT17, HiEve, and MOT20 datasets. The lack of large scale training and test data for this task has limited the community's ability to understand the performance of their tracking systems on a wide range of scenarios and conditions such as variations in person density, actions being performed, weather, and time of day. textttPersonPath22 dataset was specifically sourced to provide a wide variety of these conditions and our annotations include rich meta-data such that the performance of a tracker can be evaluated along these different dimensions. The lack of training data has also limited the ability to perform end-to-end training of tracking systems. As such, the highest performing tracking systems all rely on strong detectors trained on external image datasets. We hope that the release of this dataset will enable new lines of research that take advantage of large scale video based training data.