# Rethinking the competition between detection and ReID in Multi-Object Tracking

arXiv: https://arxiv.org/abs/2010.12138 · HTML: https://mlsanity.com/paper/2010.12138/

## Key facts

- Cited by 101 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Oct 2020.

## Built on by (most cited first, 30 of 101)

- [ByteTrack: Multi-Object Tracking by Associating Every Detection Box](https://mlsanity.com/paper/2110.06864/) (2021)
- [Observation-Centric SORT: Rethinking SORT for Robust Multi-Object Tracking](https://mlsanity.com/paper/2203.14360/) (2022)
- [BoT-SORT: Robust Associations Multi-Pedestrian Tracking](https://mlsanity.com/paper/2206.14651/) (2022)
- [StrongSORT: Make DeepSORT Great Again](https://mlsanity.com/paper/2202.13514/) (2022)
- [Deep OC-SORT: Multi-Pedestrian Tracking by Adaptive Re-Identification](https://mlsanity.com/paper/2302.11813/) (2023)
- [Global Tracking Transformers](https://mlsanity.com/paper/2203.13250/) (2022)
- [TransMOT: Spatial-Temporal Graph Transformer for Multiple Object Tracking](https://mlsanity.com/paper/2104.00194/) (2021)
- [Towards Grand Unification of Object Tracking](https://mlsanity.com/paper/2207.07078/) (2022)
- [Referring Multi-Object Tracking](https://mlsanity.com/paper/2303.03366/) (2023)
- [Hybrid-SORT: Weak Cues Matter for Online Multi-Object Tracking](https://mlsanity.com/paper/2308.00783/) (2023)
- [MotionTrack: Learning Robust Short-term and Long-term Motions for Multi-Object Tracking](https://mlsanity.com/paper/2303.10404/) (2023)
- [Do Different Tracking Tasks Require Different Appearance Models?](https://mlsanity.com/paper/2107.02156/) (2021)
- [Multiple Object Tracking as ID Prediction](https://mlsanity.com/paper/2403.16848/) (2024)
- [RelationTrack: Relation-aware Multiple Object Tracking with Decoupled Representation](https://mlsanity.com/paper/2105.04322/) (2021)
- [SparseTrack: Multi-Object Tracking by Performing Scene Decomposition based on Pseudo-Depth](https://mlsanity.com/paper/2306.05238/) (2023)
- [TransCenter: Transformers with Dense Representations for Multiple-Object Tracking](https://mlsanity.com/paper/2103.15145/) (2021)
- [Tracking Objects as Pixel-wise Distributions](https://mlsanity.com/paper/2207.05518/) (2022)
- [Towards Discriminative Representation: Multi-view Trajectory Contrastive Learning for Online Multi-object Tracking](https://mlsanity.com/paper/2203.14208/) (2022)
- [iKUN: Speak to Trackers without Retraining](https://mlsanity.com/paper/2312.16245/) (2023)
- [Quo Vadis: Is Trajectory Forecasting the Key Towards Long-Term Multi-Object Tracking?](https://mlsanity.com/paper/2210.07681/) (2022)
- [SMILEtrack: SiMIlarity LEarning for Occlusion-Aware Multiple Object Tracking](https://mlsanity.com/paper/2211.08824/) (2022)
- [One More Check: Making "Fake Background" Be Tracked Again](https://mlsanity.com/paper/2104.09441/) (2021)
- [Focus On Details: Online Multi-object Tracking with Diverse Fine-grained Representation](https://mlsanity.com/paper/2302.14589/) (2023)
- [NetTrack: Tracking Highly Dynamic Objects with a Net](https://mlsanity.com/paper/2403.11186/) (2024)
- [Bootstrapping Referring Multi-Object Tracking](https://mlsanity.com/paper/2406.05039/) (2024)
- [TOPIC: A Parallel Association Paradigm for Multi-Object Tracking under Complex Motions and Diverse Scenes](https://mlsanity.com/paper/2308.11157/) (2023)
- [ByteTrackV2: 2D and 3D Multi-Object Tracking by Associating Every Detection Box](https://mlsanity.com/paper/2303.15334/) (2023)
- [MotionTrack: Learning Motion Predictor for Multiple Object Tracking](https://mlsanity.com/paper/2306.02585/) (2023)
- [OmniTracker: Unifying Object Tracking by Tracking-with-Detection](https://mlsanity.com/paper/2303.12079/) (2023)
- [EchoTrack: Auditory Referring Multi-Object Tracking for Autonomous Driving](https://mlsanity.com/paper/2402.18302/) (2024)

## Abstract (arXiv)

Due to balanced accuracy and speed, one-shot models which jointly learn detection and identification embeddings, have drawn great attention in multi-object tracking (MOT). However, the inherent differences and relations between detection and re-identification (ReID) are unconsciously overlooked because of treating them as two isolated tasks in the one-shot tracking paradigm. This leads to inferior performance compared with existing two-stage methods. In this paper, we first dissect the reasoning process for these two tasks, which reveals that the competition between them inevitably would destroy task-dependent representations learning. To tackle this problem, we propose a novel reciprocal network (REN) with a self-relation and cross-relation design so that to impel each branch to better learn task-dependent representations. The proposed model aims to alleviate the deleterious tasks competition, meanwhile improve the cooperation between detection and ReID. Furthermore, we introduce a scale-aware attention network (SAAN) that prevents semantic level misalignment to improve the association capability of ID embeddings. By integrating the two delicately designed networks into a one-shot online MOT system, we construct a strong MOT tracker, namely CSTrack. Our tracker achieves the state-of-the-art performance on MOT16, MOT17 and MOT20 datasets, without other bells and whistles. Moreover, CSTrack is efficient and runs at 16.4 FPS on a single modern GPU, and its lightweight version even runs at 34.6 FPS. The complete code has been released at https://github.com/JudasDie/SOTS.