# Joint Learning of Siamese CNNs and Temporally Constrained Metrics for Tracklet Association

arXiv: https://arxiv.org/abs/1605.04502 · HTML: https://mlsanity.com/paper/1605.04502/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 May 2016.

## Built on by (most cited first, 3 of 3)

- [Deep Learning for UAV-based Object Detection and Tracking: A Survey](https://mlsanity.com/paper/2110.12638/) (2021)
- [Deep Learning Serves Traffic Safety Analysis: A Forward-looking Review](https://mlsanity.com/paper/2203.10939/) (2022)
- [Spatio-Temporal Point Process for Multiple Object Tracking](https://mlsanity.com/paper/2302.02444/) (2023)

## Abstract (arXiv)

In this paper, we study the challenging problem of multi-object tracking in a complex scene captured by a single camera. Different from the existing tracklet association-based tracking methods, we propose a novel and efficient way to obtain discriminative appearance-based tracklet affinity models. Our proposed method jointly learns the convolutional neural networks (CNNs) and temporally constrained metrics. In our method, a Siamese convolutional neural network (CNN) is first pre-trained on the auxiliary data. Then the Siamese CNN and temporally constrained metrics are jointly learned online to construct the appearance-based tracklet affinity models. The proposed method can jointly learn the hierarchical deep features and temporally constrained segment-wise metrics under a unified framework. For reliable association between tracklets, a novel loss function incorporating temporally constrained multi-task learning mechanism is proposed. By employing the proposed method, tracklet association can be accomplished even in challenging situations. Moreover, a new dataset with 40 fully annotated sequences is created to facilitate the tracking evaluation. Experimental results on five public datasets and the new large-scale dataset show that our method outperforms several state-of-the-art approaches in multi-object tracking.