# Multi-Object Tracking with Siamese Track-RCNN

arXiv: https://arxiv.org/abs/2004.07786 · HTML: https://mlsanity.com/paper/2004.07786/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Apr 2020.

## Built on by (most cited first, 14 of 14)

- [MOTR: End-to-End Multiple-Object Tracking with Transformer](https://mlsanity.com/paper/2105.03247/) (2021)
- [MOTRv2: Bootstrapping End-to-End Multi-Object Tracking by Pretrained Object Detectors](https://mlsanity.com/paper/2211.09791/) (2022)
- [Deep Learning for UAV-based Object Detection and Tracking: A Survey](https://mlsanity.com/paper/2110.12638/) (2021)
- [Semi-TCL: Semi-Supervised Track Contrastive Representation Learning](https://mlsanity.com/paper/2107.02396/) (2021)
- [City-Scale Multi-Camera Vehicle Tracking Guided by Crossroad Zones](https://mlsanity.com/paper/2105.06623/) (2021)
- [A topological solution to object segmentation and tracking](https://mlsanity.com/paper/2107.02036/) (2021)
- [Semi-Automated Computer Vision based Tracking of Multiple Industrial Entities -- A Framework and Dataset Creation Approach](https://mlsanity.com/paper/2304.00950/) (2023)
- [DroneMOT: Drone-based Multi-Object Tracking Considering Detection Difficulties and Simultaneous Moving of Drones and Objects](https://mlsanity.com/paper/2407.09051/) (2024)
- [To New Beginnings: A Survey of Unified Perception in Autonomous Vehicle Software](https://mlsanity.com/paper/2508.20892/) (2025)
- [Multi-object Tracking with a Hierarchical Single-branch Network](https://mlsanity.com/paper/2101.01984/) (2021)
- [Merging Tasks for Video Panoptic Segmentation](https://mlsanity.com/paper/2108.04223/) (2021)
- [A Hybrid Approach To Real-Time Multi-Object Tracking](https://mlsanity.com/paper/2308.01248/) (2023)
- [VSE-MOT: Multi-Object Tracking in Low-Quality Video Scenes Guided by Visual Semantic Enhancement](https://mlsanity.com/paper/2509.14060/) (2025)
- [VSD-MOT: End-to-End Multi-Object Tracking in Low-Quality Video Scenes Guided by Visual Semantic Distillation](https://mlsanity.com/paper/2603.20731/) (2026)

## Abstract (arXiv)

Multi-object tracking systems often consist of a combination of a detector, a short term linker, a re-identification feature extractor and a solver that takes the output from these separate components and makes a final prediction. Differently, this work aims to unify all these in a single tracking system. Towards this, we propose Siamese Track-RCNN, a two stage detect-and-track framework which consists of three functional branches: (1) the detection branch localizes object instances; (2) the Siamese-based track branch estimates the object motion and (3) the object re-identification branch re-activates the previously terminated tracks when they re-emerge. We test our tracking system on two popular datasets of the MOTChallenge. Siamese Track-RCNN achieves significantly higher results than the state-of-the-art, while also being much more efficient, thanks to its unified design.