# Samba: Synchronized Set-of-Sequences Modeling for Multiple Object Tracking

arXiv: https://arxiv.org/abs/2410.01806 · HTML: https://mlsanity.com/paper/2410.01806/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- Builds on 52 of its 59 references that are in the map.
- First posted on arXiv 2 Oct 2024.

## Builds on (most cited first, 30 of 52)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Layer Normalization](https://mlsanity.com/paper/1607.06450/) (2016)
- [Mamba: Linear-Time Sequence Modeling with Selective State Spaces](https://mlsanity.com/paper/2312.00752/) (2023)
- [Gaussian Error Linear Units (GELUs)](https://mlsanity.com/paper/1606.08415/) (2016)
- [Deformable DETR: Deformable Transformers for End-to-End Object Detection](https://mlsanity.com/paper/2010.04159/) (2020)
- [Efficiently Modeling Long Sequences with Structured State Spaces](https://mlsanity.com/paper/2111.00396/) (2021)
- [Root Mean Square Layer Normalization](https://mlsanity.com/paper/1910.07467/) (2019)
- [YOLOX: Exceeding YOLO Series in 2021](https://mlsanity.com/paper/2107.08430/) (2021)
- [Simple Online and Realtime Tracking with a Deep Association Metric](https://mlsanity.com/paper/1703.07402/) (2017)
- [Simple Online and Realtime Tracking](https://mlsanity.com/paper/1602.00763/) (2016)
- [ByteTrack: Multi-Object Tracking by Associating Every Detection Box](https://mlsanity.com/paper/2110.06864/) (2021)
- [Performance Measures and a Data Set for Multi-Target, Multi-Camera Tracking](https://mlsanity.com/paper/1609.01775/) (2016)
- [CenterNet: Keypoint Triplets for Object Detection](https://mlsanity.com/paper/1904.08189/) (2019)
- [DAB-DETR: Dynamic Anchor Boxes are Better Queries for DETR](https://mlsanity.com/paper/2201.12329/) (2022)
- [HOTA: A Higher Order Metric for Evaluating Multi-Object Tracking](https://mlsanity.com/paper/2009.07736/) (2020)
- [MOT16: A Benchmark for Multi-Object Tracking](https://mlsanity.com/paper/1603.00831/) (2016)
- [TrackFormer: Multi-Object Tracking with Transformers](https://mlsanity.com/paper/2101.02702/) (2021)
- [Transformers in Time Series: A Survey](https://mlsanity.com/paper/2202.07125/) (2022)
- [FairMOT: On the Fairness of Detection and Re-Identification in Multiple Object Tracking](https://mlsanity.com/paper/2004.01888/) (2020)
- [Tracking Objects as Points](https://mlsanity.com/paper/2004.01177/) (2020)
- [MOTR: End-to-End Multiple-Object Tracking with Transformer](https://mlsanity.com/paper/2105.03247/) (2021)
- [TransTrack: Multiple Object Tracking with Transformer](https://mlsanity.com/paper/2012.15460/) (2020)
- [Observation-Centric SORT: Rethinking SORT for Robust Multi-Object Tracking](https://mlsanity.com/paper/2203.14360/) (2022)
- [Towards Real-Time Multi-Object Tracking](https://mlsanity.com/paper/1909.12605/) (2019)
- [Quasi-Dense Similarity Learning for Multiple Object Tracking](https://mlsanity.com/paper/2006.06664/) (2020)
- [DanceTrack: Multi-Object Tracking in Uniform Appearance and Diverse Motion](https://mlsanity.com/paper/2111.14690/) (2021)
- [Multiple Object Tracking: A Literature Review](https://mlsanity.com/paper/1409.7618/) (2014)

## Built on by (most cited first, 21 of 21)

- [StateSpaceDiffuser: Bringing Long Context to Diffusion World Models](https://mlsanity.com/paper/2505.22246/) (2025)
- [History-Aware Transformation of ReID Features for Multiple Object Tracking](https://mlsanity.com/paper/2503.12562/) (2025)
- [AR-MOT: Autoregressive Multi-object Tracking](https://mlsanity.com/paper/2601.01925/) (2026)
- [Dual-level Adaptation for Multi-Object Tracking: Building Test-Time Calibration from Experience and Intuition](https://mlsanity.com/paper/2603.21629/) (2026)
- [Hypergraph-State Collaborative Reasoning for Multi-Object Tracking](https://mlsanity.com/paper/2604.12665/) (2026)
- [To New Beginnings: A Survey of Unified Perception in Autonomous Vehicle Software](https://mlsanity.com/paper/2508.20892/) (2025)
- [GenTrack: A New Generation of Multi-Object Tracking](https://mlsanity.com/paper/2510.24399/) (2025)
- [From Detection to Association: Learning Discriminative Object Embeddings for Multi-Object Tracking](https://mlsanity.com/paper/2512.02392/) (2025)
- [Clip-level Uncertainty and Temporal-aware Active Learning for End-to-End Multi-Object Tracking](https://mlsanity.com/paper/2605.09858/) (2026)
- [Polycepta: Object-Centric Appearance Estimation for Multi-Object Tracking](https://mlsanity.com/paper/2606.23604/) (2026)
- [SAM2Auto: Auto Annotation Using FLASH](https://mlsanity.com/paper/2506.07850/) (2025)
- [SelfMOTR: Revisiting MOTR with Self-Generating Detection Priors](https://mlsanity.com/paper/2511.20279/) (2025)
- [Training-Free Global Geometric Association for 4D LiDAR Panoptic Segmentation](https://mlsanity.com/paper/2512.18991/) (2025)
- [Motion-Driven Multi-Object Tracking of Model Organisms in Space Science Experiments](https://mlsanity.com/paper/2604.26321/) (2026)
- [Training-Free Long-Term Multi-Object Tracking for Sports Video Analytics](https://mlsanity.com/paper/2608.15688/) (2026)
- [Probing Association Instability with Track-State Perturbations for Clip-Level Active Learning in Query-Propagation Multi-Object Tracking](https://mlsanity.com/paper/2608.17224/) (2026)
- [Beyond the Image Plane: World-Grounded Queries for Multi-Object Tracking](https://mlsanity.com/paper/2609.00924/) (2026)
- [Beyond the Survey: A Systematic Empirical Study of Detection and Association in Visual MOT](https://mlsanity.com/paper/2609.22291/) (2026)
- [ReWorld-Track: A Recursive Event World Model for Language-Guided Multi-Camera Tracking](https://mlsanity.com/paper/2609.36677/) (2026)
- [TrackFish3D: Self-Supervised 3D Tracking of Schooling Fish from Multi-view Videos](https://mlsanity.com/paper/2609.38347/) (2026)
- [SceneForesee: Learning Scene Evolution for Spatiotemporal Reasoning](https://mlsanity.com/paper/2610.14439/) (2026)

## Abstract (arXiv)

Multiple object tracking in complex scenarios - such as coordinated dance performances, team sports, or dynamic animal groups - presents unique challenges. In these settings, objects frequently move in coordinated patterns, occlude each other, and exhibit long-term dependencies in their trajectories. However, it remains a key open research question on how to model long-range dependencies within tracklets, interdependencies among tracklets, and the associated temporal occlusions. To this end, we introduce Samba, a novel linear-time set-of-sequences model designed to jointly process multiple tracklets by synchronizing the multiple selective state-spaces used to model each tracklet. Samba autoregressively predicts the future track query for each sequence while maintaining synchronized long-term memory representations across tracklets. By integrating Samba into a tracking-by-propagation framework, we propose SambaMOTR, the first tracker effectively addressing the aforementioned issues, including long-range dependencies, tracklet interdependencies, and temporal occlusions. Additionally, we introduce an effective technique for dealing with uncertain observations (MaskObs) and an efficient training recipe to scale SambaMOTR to longer sequences. By modeling long-range dependencies and interactions among tracked objects, SambaMOTR implicitly learns to track objects accurately through occlusions without any hand-crafted heuristics. Our approach significantly surpasses prior state-of-the-art on the DanceTrack, BFT, and SportsMOT datasets.