# Online Multi-Object Tracking Using CNN-based Single Object Tracker with Spatial-Temporal Attention Mechanism

arXiv: https://arxiv.org/abs/1708.02843 · HTML: https://mlsanity.com/paper/1708.02843/

## Key facts

- Cited by 48 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 Aug 2017.

## Built on by (most cited first, 30 of 48)

- [TrackFormer: Multi-Object Tracking with Transformers](https://mlsanity.com/paper/2101.02702/) (2021)
- [MOSE: A New Dataset for Video Object Segmentation in Complex Scenes](https://mlsanity.com/paper/2302.01872/) (2023)
- [Occluded Video Instance Segmentation: A Benchmark](https://mlsanity.com/paper/2102.01558/) (2021)
- [TransMOT: Spatial-Temporal Graph Transformer for Multiple Object Tracking](https://mlsanity.com/paper/2104.00194/) (2021)
- [MOTSynth: How Can Synthetic Data Help Pedestrian Detection and Tracking?](https://mlsanity.com/paper/2108.09518/) (2021)
- [MOSEv2: A More Challenging Dataset for Video Object Segmentation in Complex Scenes](https://mlsanity.com/paper/2508.05630/) (2025)
- [Attention, please! A survey of Neural Attention Models in Deep Learning](https://mlsanity.com/paper/2103.16775/) (2021)
- [Deep Learning for UAV-based Object Detection and Tracking: A Survey](https://mlsanity.com/paper/2110.12638/) (2021)
- [One More Check: Making "Fake Background" Be Tracked Again](https://mlsanity.com/paper/2104.09441/) (2021)
- [Discriminative Appearance Modeling with Multi-track Pooling for Real-time Multi-object Tracking](https://mlsanity.com/paper/2101.12159/) (2021)
- [Multi-Camera Multiple 3D Object Tracking on the Move for Autonomous Vehicles](https://mlsanity.com/paper/2204.09151/) (2022)
- [EmoNet: A Transfer Learning Framework for Multi-Corpus Speech Emotion Recognition](https://mlsanity.com/paper/2103.08310/) (2021)
- [Online Multi-Object Tracking with Unsupervised Re-Identification Learning and Occlusion Estimation](https://mlsanity.com/paper/2201.01297/) (2022)
- [Artificial Intelligence for Satellite Communication: A Review](https://mlsanity.com/paper/2101.10899/) (2021)
- [SOTVerse: A User-defined Task Space of Single Object Tracking](https://mlsanity.com/paper/2204.07414/) (2022)
- [Know Your Surroundings: Panoramic Multi-Object Tracking by Multimodality Collaboration](https://mlsanity.com/paper/2105.14683/) (2021)
- [Tracking by Joint Local and Global Search: A Target-aware Attention based Approach](https://mlsanity.com/paper/2106.04840/) (2021)
- [Multi-Camera Multi-Object Tracking on the Move via Single-Stage Global Association Approach](https://mlsanity.com/paper/2211.09663/) (2022)
- [Real-time Online Multi-Object Tracking in Compressed Domain](https://mlsanity.com/paper/2204.02081/) (2022)
- [Towards Frame Rate Agnostic Multi-Object Tracking](https://mlsanity.com/paper/2209.11404/) (2022)
- [Spatio-Temporal Point Process for Multiple Object Tracking](https://mlsanity.com/paper/2302.02444/) (2023)
- [An Explainable Non-local Network for COVID-19 Diagnosis](https://mlsanity.com/paper/2408.04300/) (2024)
- [ConCAD: Contrastive Learning-based Cross Attention for Sleep Apnea Detection](https://mlsanity.com/paper/2105.03037/) (2021)
- [You Better Look Twice: a new perspective for designing accurate detectors with reduced computations](https://mlsanity.com/paper/2107.10050/) (2021)
- [Spatial-Temporal Map Vehicle Trajectory Detection Using Dynamic Mode Decomposition and Res-UNet+ Neural Networks](https://mlsanity.com/paper/2201.04755/) (2022)
- [DSRRTracker: Dynamic Search Region Refinement for Attention-based Siamese Multi-Object Tracking](https://mlsanity.com/paper/2203.10729/) (2022)
- [Transformer-based assignment decision network for multiple object tracking](https://mlsanity.com/paper/2208.03571/) (2022)
- [RLM-Tracking: Online Multi-Pedestrian Tracking Supported by Relative Location Mapping](https://mlsanity.com/paper/2210.10477/) (2022)
- [Graph-Based Multi-Camera Soccer Player Tracker](https://mlsanity.com/paper/2211.02125/) (2022)
- [Joint Counting, Detection and Re-Identification for Multi-Object Tracking](https://mlsanity.com/paper/2212.05861/) (2022)

## Abstract (arXiv)

In this paper, we propose a CNN-based framework for online MOT. This framework utilizes the merits of single object trackers in adapting appearance models and searching for target in the next frame. Simply applying single object tracker for MOT will encounter the problem in computational efficiency and drifted results caused by occlusion. Our framework achieves computational efficiency by sharing features and using ROI-Pooling to obtain individual features for each target. Some online learned target-specific CNN layers are used for adapting the appearance model for each target. In the framework, we introduce spatial-temporal attention mechanism (STAM) to handle the drift caused by occlusion and interaction among targets. The visibility map of the target is learned and used for inferring the spatial attention map. The spatial attention map is then applied to weight the features. Besides, the occlusion status can be estimated from the visibility map, which controls the online updating process via weighted loss on training samples with different occlusion statuses in different frames. It can be considered as temporal attention mechanism. The proposed algorithm achieves 34.3% and 46.0% in MOTA on challenging MOT15 and MOT16 benchmark dataset respectively.