# Real-time Multiple People Tracking with Deeply Learned Candidate Selection and Person Re-Identification

arXiv: https://arxiv.org/abs/1809.04427 · HTML: https://mlsanity.com/paper/1809.04427/

## Key facts

- Cited by 96 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Sep 2018.

## Built on by (most cited first, 30 of 96)

- [ByteTrack: Multi-Object Tracking by Associating Every Detection Box](https://mlsanity.com/paper/2110.06864/) (2021)
- [TrackFormer: Multi-Object Tracking with Transformers](https://mlsanity.com/paper/2101.02702/) (2021)
- [MOTR: End-to-End Multiple-Object Tracking with Transformer](https://mlsanity.com/paper/2105.03247/) (2021)
- [DanceTrack: Multi-Object Tracking in Uniform Appearance and Diverse Motion](https://mlsanity.com/paper/2111.14690/) (2021)
- [StrongSORT: Make DeepSORT Great Again](https://mlsanity.com/paper/2202.13514/) (2022)
- [MOTRv2: Bootstrapping End-to-End Multi-Object Tracking by Pretrained Object Detectors](https://mlsanity.com/paper/2211.09791/) (2022)
- [Multiple Object Tracking with Correlation Learning](https://mlsanity.com/paper/2104.03541/) (2021)
- [Hybrid-SORT: Weak Cues Matter for Online Multi-Object Tracking](https://mlsanity.com/paper/2308.00783/) (2023)
- [Do Different Tracking Tasks Require Different Appearance Models?](https://mlsanity.com/paper/2107.02156/) (2021)
- [Multiple Object Tracking as ID Prediction](https://mlsanity.com/paper/2403.16848/) (2024)
- [DEFT: Detection Embeddings for Tracking](https://mlsanity.com/paper/2102.02267/) (2021)
- [RelationTrack: Relation-aware Multiple Object Tracking with Decoupled Representation](https://mlsanity.com/paper/2105.04322/) (2021)
- [SparseTrack: Multi-Object Tracking by Performing Scene Decomposition based on Pseudo-Depth](https://mlsanity.com/paper/2306.05238/) (2023)
- [Simple Cues Lead to a Strong Multi-Object Tracker](https://mlsanity.com/paper/2206.04656/) (2022)
- [TransCenter: Transformers with Dense Representations for Multiple-Object Tracking](https://mlsanity.com/paper/2103.15145/) (2021)
- [DiffusionTrack: Diffusion Model For Multi-Object Tracking](https://mlsanity.com/paper/2308.09905/) (2023)
- [VoxelTrack: Multi-Person 3D Human Pose Estimation and Tracking in the Wild](https://mlsanity.com/paper/2108.02452/) (2021)
- [EarlyBird: Early-Fusion for Multi-View Tracking in the Bird's Eye View](https://mlsanity.com/paper/2310.13350/) (2023)
- [One More Check: Making "Fake Background" Be Tracked Again](https://mlsanity.com/paper/2104.09441/) (2021)
- [Lifting Multi-View Detection and Tracking to the Bird's Eye View](https://mlsanity.com/paper/2403.12573/) (2024)
- [Discriminative Appearance Modeling with Multi-track Pooling for Real-time Multi-object Tracking](https://mlsanity.com/paper/2101.12159/) (2021)
- [Regularity Learning via Explicit Distribution Modeling for Skeletal Video Anomaly Detection](https://mlsanity.com/paper/2112.03649/) (2021)
- [Looking Beyond Two Frames: End-to-End Multi-Object Tracking Using Spatial and Temporal Transformers](https://mlsanity.com/paper/2103.14829/) (2021)
- [ByteTrackV2: 2D and 3D Multi-Object Tracking by Associating Every Detection Box](https://mlsanity.com/paper/2303.15334/) (2023)
- [MotionTrack: Learning Motion Predictor for Multiple Object Tracking](https://mlsanity.com/paper/2306.02585/) (2023)
- [MambaTrack: A Simple Baseline for Multiple Object Tracking with State Space Model](https://mlsanity.com/paper/2408.09178/) (2024)
- [Detection Recovery in Online Multi-Object Tracking with Sparse Graph Tracker](https://mlsanity.com/paper/2205.00968/) (2022)
- [Person Recognition at Altitude and Range: Fusion of Face, Body Shape and Gait](https://mlsanity.com/paper/2505.04616/) (2025)
- [Online Multiple Object Tracking with Cross-Task Synergy](https://mlsanity.com/paper/2104.00380/) (2021)
- [Uncertainty-aware Unsupervised Multi-Object Tracking](https://mlsanity.com/paper/2307.15409/) (2023)

## Abstract (arXiv)

Online multi-object tracking is a fundamental problem in time-critical video analysis applications. A major challenge in the popular tracking-by-detection framework is how to associate unreliable detection results with existing tracks. In this paper, we propose to handle unreliable detection by collecting candidates from outputs of both detection and tracking. The intuition behind generating redundant candidates is that detection and tracks can complement each other in different scenarios. Detection results of high confidence prevent tracking drifts in the long term, and predictions of tracks can handle noisy detection caused by occlusion. In order to apply optimal selection from a considerable amount of candidates in real-time, we present a novel scoring function based on a fully convolutional neural network, that shares most computations on the entire image. Moreover, we adopt a deeply learned appearance representation, which is trained on large-scale person re-identification datasets, to improve the identification ability of our tracker. Extensive experiments show that our tracker achieves real-time and state-of-the-art performance on a widely used people tracking benchmark.