# On the detection-to-track association for online multi-object tracking

arXiv: https://arxiv.org/abs/2107.00500 · HTML: https://mlsanity.com/paper/2107.00500/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 34 references that are in the map.
- First posted on arXiv 1 Jul 2021.

## Builds on (most cited first, 13 of 13)

- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [YOLOv4: Optimal Speed and Accuracy of Object Detection](https://mlsanity.com/paper/2004.10934/) (2020)
- [Simple Online and Realtime Tracking with a Deep Association Metric](https://mlsanity.com/paper/1703.07402/) (2017)
- [Simple Online and Realtime Tracking](https://mlsanity.com/paper/1602.00763/) (2016)
- [Performance Measures and a Data Set for Multi-Target, Multi-Camera Tracking](https://mlsanity.com/paper/1609.01775/) (2016)
- [MOT16: A Benchmark for Multi-Object Tracking](https://mlsanity.com/paper/1603.00831/) (2016)
- [Towards Real-Time Multi-Object Tracking](https://mlsanity.com/paper/1909.12605/) (2019)
- [CrowdHuman: A Benchmark for Detecting Human in a Crowd](https://mlsanity.com/paper/1805.00123/) (2018)
- [Person Re-identification in the Wild](https://mlsanity.com/paper/1604.02531/) (2016)
- [MOTChallenge 2015: Towards a Benchmark for Multi-Target Tracking](https://mlsanity.com/paper/1504.01942/) (2015)
- [POI: Multiple Object Tracking with High Performance Detection and Appearance Feature](https://mlsanity.com/paper/1610.06136/) (2016)
- [A Fast Incremental Gaussian Mixture Model](https://mlsanity.com/paper/1506.04422/) (2015)
- [SQE: a Self Quality Evaluation Metric for Parameters Optimization in Multi-Object Tracking](https://mlsanity.com/paper/2004.07472/) (2020)

## Built on by (most cited first, 3 of 3)

- [PatchTrack: Multiple Object Tracking Using Frame Patches](https://mlsanity.com/paper/2201.00080/) (2022)
- [Joint Learning Architecture for Multiple Object Tracking and Trajectory Forecasting](https://mlsanity.com/paper/2108.10543/) (2021)
- [Sparse3DTrack: Monocular 3D Object Tracking Using Sparse Supervision](https://mlsanity.com/paper/2603.18298/) (2026)

## Abstract (arXiv)

Driven by recent advances in object detection with deep neural networks, the tracking-by-detection paradigm has gained increasing prevalence in the research community of multi-object tracking (MOT). It has long been known that appearance information plays an essential role in the detection-to-track association, which lies at the core of the tracking-by-detection paradigm. While most existing works consider the appearance distances between the detections and the tracks, they ignore the statistical information implied by the historical appearance distance records in the tracks, which can be particularly useful when a detection has similar distances with two or more tracks. In this work, we propose a hybrid track association (HTA) algorithm that models the historical appearance distances of a track with an incremental Gaussian mixture model (IGMM) and incorporates the derived statistical information into the calculation of the detection-to-track association cost. Experimental results on three MOT benchmarks confirm that HTA effectively improves the target identification performance with a small compromise to the tracking speed. Additionally, compared to many state-of-the-art trackers, the DeepSORT tracker equipped with HTA achieves better or comparable performance in terms of the balance of tracking quality and speed.