# SMILEtrack: SiMIlarity LEarning for Occlusion-Aware Multiple Object Tracking

arXiv: https://arxiv.org/abs/2211.08824 · HTML: https://mlsanity.com/paper/2211.08824/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- Builds on 46 of its 58 references that are in the map.
- First posted on arXiv 16 Nov 2022.

## Builds on (most cited first, 30 of 46)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [You Only Look Once: Unified, Real-Time Object Detection](https://mlsanity.com/paper/1506.02640/) (2015)
- [Deformable DETR: Deformable Transformers for End-to-End Object Detection](https://mlsanity.com/paper/2010.04159/) (2020)
- [SSD: Single Shot MultiBox Detector](https://mlsanity.com/paper/1512.02325/) (2015)
- [YOLOv3: An Incremental Improvement](https://mlsanity.com/paper/1804.02767/) (2018)
- [YOLOv4: Optimal Speed and Accuracy of Object Detection](https://mlsanity.com/paper/2004.10934/) (2020)
- [YOLO9000: Better, Faster, Stronger](https://mlsanity.com/paper/1612.08242/) (2016)
- [Objects as Points](https://mlsanity.com/paper/1904.07850/) (2019)
- [YOLOX: Exceeding YOLO Series in 2021](https://mlsanity.com/paper/2107.08430/) (2021)
- [Simple Online and Realtime Tracking with a Deep Association Metric](https://mlsanity.com/paper/1703.07402/) (2017)
- [Simple Online and Realtime Tracking](https://mlsanity.com/paper/1602.00763/) (2016)
- [ByteTrack: Multi-Object Tracking by Associating Every Detection Box](https://mlsanity.com/paper/2110.06864/) (2021)
- [Performance Measures and a Data Set for Multi-Target, Multi-Camera Tracking](https://mlsanity.com/paper/1609.01775/) (2016)
- [HOTA: A Higher Order Metric for Evaluating Multi-Object Tracking](https://mlsanity.com/paper/2009.07736/) (2020)
- [MOT16: A Benchmark for Multi-Object Tracking](https://mlsanity.com/paper/1603.00831/) (2016)
- [TrackFormer: Multi-Object Tracking with Transformers](https://mlsanity.com/paper/2101.02702/) (2021)
- [FairMOT: On the Fairness of Detection and Re-Identification in Multiple Object Tracking](https://mlsanity.com/paper/2004.01888/) (2020)
- [Tracking Objects as Points](https://mlsanity.com/paper/2004.01177/) (2020)
- [CSPNet: A New Backbone that can Enhance Learning Capability of CNN](https://mlsanity.com/paper/1911.11929/) (2019)
- [MOTR: End-to-End Multiple-Object Tracking with Transformer](https://mlsanity.com/paper/2105.03247/) (2021)
- [TransTrack: Multiple Object Tracking with Transformer](https://mlsanity.com/paper/2012.15460/) (2020)
- [MOT20: A benchmark for multi object tracking in crowded scenes](https://mlsanity.com/paper/2003.09003/) (2020)
- [Observation-Centric SORT: Rethinking SORT for Robust Multi-Object Tracking](https://mlsanity.com/paper/2203.14360/) (2022)
- [BoT-SORT: Robust Associations Multi-Pedestrian Tracking](https://mlsanity.com/paper/2206.14651/) (2022)
- [Towards Real-Time Multi-Object Tracking](https://mlsanity.com/paper/1909.12605/) (2019)
- [CrowdHuman: A Benchmark for Detecting Human in a Crowd](https://mlsanity.com/paper/1805.00123/) (2018)
- [Quasi-Dense Similarity Learning for Multiple Object Tracking](https://mlsanity.com/paper/2006.06664/) (2020)

## Built on by (most cited first, 29 of 29)

- [OmniTracker: Unifying Object Tracking by Tracking-with-Detection](https://mlsanity.com/paper/2303.12079/) (2023)
- [CholecTrack20: A Multi-Perspective Tracking Dataset for Surgical Tools](https://mlsanity.com/paper/2312.07352/) (2023)
- [BoostTrack++: using tracklet information to detect more objects in multiple object tracking](https://mlsanity.com/paper/2408.13003/) (2024)
- [Tracking one-in-a-million: Large-scale benchmark for microbial single-cell tracking with experiment-aware robustness metrics](https://mlsanity.com/paper/2411.00552/) (2024)
- [Weakly Supervised Video Individual CountingWeakly Supervised Video Individual Counting](https://mlsanity.com/paper/2312.05923/) (2023)
- [Vision Technologies with Applications in Traffic Surveillance Systems: A Holistic Survey](https://mlsanity.com/paper/2412.00348/) (2024)
- [Deep Learning-Based Multi-Object Tracking: A Comprehensive Survey from Foundations to State-of-the-Art](https://mlsanity.com/paper/2506.13457/) (2025)
- [SurgiTrack: Fine-Grained Multi-Class Multi-Tool Tracking in Surgical Videos](https://mlsanity.com/paper/2405.20333/) (2024)
- [YOLO11-JDE: Fast and Accurate Multi-Object Tracking with Self-Supervised Re-ID](https://mlsanity.com/paper/2501.13710/) (2025)
- [RoundaboutHD: High-Resolution Real-World Urban Environment Benchmark for Multi-Camera Vehicle Tracking](https://mlsanity.com/paper/2507.08729/) (2025)
- [SFSORT: Scene Features-based Simple Online Real-Time Tracker](https://mlsanity.com/paper/2404.07553/) (2024)
- [Multi Player Tracking in Ice Hockey with Homographic Projections](https://mlsanity.com/paper/2405.13397/) (2024)
- [Tracking the Unstable: Appearance-Guided Motion Modeling for Robust Multi-Object Tracking in UAV-Captured Videos](https://mlsanity.com/paper/2508.01730/) (2025)
- [A Multi-purpose Tracking Framework for Salmon Welfare Monitoring in Challenging Environments](https://mlsanity.com/paper/2509.25969/) (2025)
- [Multi-Camera Worker Tracking in Logistics Warehouse Considering Wide-Angle Distortion](https://mlsanity.com/paper/2510.19432/) (2025)
- [GenTrack: A New Generation of Multi-Object Tracking](https://mlsanity.com/paper/2510.24399/) (2025)
- [GenTrack2: An Improved Hybrid Approach for Multi-Object Tracking](https://mlsanity.com/paper/2510.24410/) (2025)
- [TransientTrack: Advanced Multi-Object Tracking and Classification of Cancer Cells with Transient Fluorescent Signals](https://mlsanity.com/paper/2512.01885/) (2025)
- [Occlusion-Aware SORT: Observing Occlusion for Robust Multi-Object Tracking](https://mlsanity.com/paper/2603.06034/) (2026)
- [Gr-IoU: Ground-Intersection over Union for Robust Multi-Object Tracking with 3D Geometric Constraints](https://mlsanity.com/paper/2409.03252/) (2024)
- [When to Extract ReID Features: A Selective Approach for Improved Multiple Object Tracking](https://mlsanity.com/paper/2409.06617/) (2024)
- [Enhanced Transformer-Based Tracking for Skiing Events: Overcoming Multi-Camera Challenges, Scale Variations and Rapid Motion -- SkiTB Visual Tracking Challenge 2025](https://mlsanity.com/paper/2502.18867/) (2025)
- [SMTrack: End-to-End Trained Spiking Neural Networks for Multi-Object Tracking in RGB Videos](https://mlsanity.com/paper/2508.14607/) (2025)
- [CorVS+: Correspondence-Driven Association of Video Trajectories and Sensors for Identity-Aware Person Localization in Warehouses](https://mlsanity.com/paper/2510.26369/) (2025)
- [FC-Track: Overlap-Aware Post-Association Correction for Online Multi-Object Tracking](https://mlsanity.com/paper/2603.12758/) (2026)
- [SAMOFT: Robust Multi-Object Tracking via Region and Flow](https://mlsanity.com/paper/2605.09417/) (2026)
- [MicroViTv2: Beyond the FLOPS for Edge Energy-Friendly Vision Transformers](https://mlsanity.com/paper/2605.10148/) (2026)
- [CoDAT: Collaborative Dual-Attention Transformer with Low-Cost Temporal Modeling for Efficient Edge Action Recognition](https://mlsanity.com/paper/2608.06691/) (2026)
- [GenTrack3: Hybrid Stochastic-Deterministic Online Multi-Object Tracking with Cluster-Aware Association](https://mlsanity.com/paper/2608.09581/) (2026)

## Abstract (arXiv)

Despite recent progress in Multiple Object Tracking (MOT), several obstacles such as occlusions, similar objects, and complex scenes remain an open challenge. Meanwhile, a systematic study of the cost-performance tradeoff for the popular tracking-by-detection paradigm is still lacking. This paper introduces SMILEtrack, an innovative object tracker that effectively addresses these challenges by integrating an efficient object detector with a Siamese network-based Similarity Learning Module (SLM). The technical contributions of SMILETrack are twofold. First, we propose an SLM that calculates the appearance similarity between two objects, overcoming the limitations of feature descriptors in Separate Detection and Embedding (SDE) models. The SLM incorporates a Patch Self-Attention (PSA) block inspired by the vision Transformer, which generates reliable features for accurate similarity matching. Second, we develop a Similarity Matching Cascade (SMC) module with a novel GATE function for robust object matching across consecutive video frames, further enhancing MOT performance. Together, these innovations help SMILETrack achieve an improved trade-off between the cost (em e.g., running speed) and performance (e.g., tracking accuracy) over several existing state-of-the-art benchmarks, including the popular BYTETrack method. SMILETrack outperforms BYTETrack by 0.4-0.8 MOTA and 2.1-2.2 HOTA points on MOT17 and MOT20 datasets. Code is available at https://github.com/pingyang1117/SMILEtrackOfficial