# TrackSSM: A General Motion Predictor by State-Space Model arXiv: https://arxiv.org/abs/2409.00487 ยท HTML: https://mlsanity.com/paper/2409.00487/ ## Key facts - Cited by 6 papers in the map (data through 14 Oct 2026). - Builds on 50 of its 55 references that are in the map. - First posted on arXiv 31 Aug 2024. ## Builds on (most cited first, 30 of 50) - [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017) - [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020) - [Diffusion Models Beat GANs on Image Synthesis](https://mlsanity.com/paper/2105.05233/) (2021) - [Mamba: Linear-Time Sequence Modeling with Selective State Spaces](https://mlsanity.com/paper/2312.00752/) (2023) - [Efficiently Modeling Long Sequences with Structured State Spaces](https://mlsanity.com/paper/2111.00396/) (2021) - [Rich feature hierarchies for accurate object detection and semantic segmentation](https://mlsanity.com/paper/1311.2524/) (2013) - [Generalized Intersection over Union: A Metric and A Loss for Bounding Box Regression](https://mlsanity.com/paper/1902.09630/) (2019) - [BDD100K: A Diverse Driving Dataset for Heterogeneous Multitask Learning](https://mlsanity.com/paper/1805.04687/) (2018) - [YOLOX: Exceeding YOLO Series in 2021](https://mlsanity.com/paper/2107.08430/) (2021) - [Simple Online and Realtime Tracking with a Deep Association Metric](https://mlsanity.com/paper/1703.07402/) (2017) - [Simple Online and Realtime Tracking](https://mlsanity.com/paper/1602.00763/) (2016) - [ByteTrack: Multi-Object Tracking by Associating Every Detection Box](https://mlsanity.com/paper/2110.06864/) (2021) - [Simplified State Space Layers for Sequence Modeling](https://mlsanity.com/paper/2208.04933/) (2022) - [Performance Measures and a Data Set for Multi-Target, Multi-Camera Tracking](https://mlsanity.com/paper/1609.01775/) (2016) - [HOTA: A Higher Order Metric for Evaluating Multi-Object Tracking](https://mlsanity.com/paper/2009.07736/) (2020) - [MOT16: A Benchmark for Multi-Object Tracking](https://mlsanity.com/paper/1603.00831/) (2016) - [TrackFormer: Multi-Object Tracking with Transformers](https://mlsanity.com/paper/2101.02702/) (2021) - [FairMOT: On the Fairness of Detection and Re-Identification in Multiple Object Tracking](https://mlsanity.com/paper/2004.01888/) (2020) - [Tracking Objects as Points](https://mlsanity.com/paper/2004.01177/) (2020) - [Tracking without bells and whistles](https://mlsanity.com/paper/1903.05625/) (2019) - [MOTR: End-to-End Multiple-Object Tracking with Transformer](https://mlsanity.com/paper/2105.03247/) (2021) - [TransTrack: Multiple Object Tracking with Transformer](https://mlsanity.com/paper/2012.15460/) (2020) - [MOT20: A benchmark for multi object tracking in crowded scenes](https://mlsanity.com/paper/2003.09003/) (2020) - [Observation-Centric SORT: Rethinking SORT for Robust Multi-Object Tracking](https://mlsanity.com/paper/2203.14360/) (2022) - [BoT-SORT: Robust Associations Multi-Pedestrian Tracking](https://mlsanity.com/paper/2206.14651/) (2022) - [Towards Real-Time Multi-Object Tracking](https://mlsanity.com/paper/1909.12605/) (2019) - [Omni-Scale Feature Learning for Person Re-Identification](https://mlsanity.com/paper/1905.00953/) (2019) - [Quasi-Dense Similarity Learning for Multiple Object Tracking](https://mlsanity.com/paper/2006.06664/) (2020) - [DanceTrack: Multi-Object Tracking in Uniform Appearance and Diverse Motion](https://mlsanity.com/paper/2111.14690/) (2021) - [StrongSORT: Make DeepSORT Great Again](https://mlsanity.com/paper/2202.13514/) (2022) ## Built on by (most cited first, 6 of 6) - [SportMamba: Adaptive Non-Linear Multi-Object Tracking with State Space Models for Team Sports](https://mlsanity.com/paper/2506.03335/) (2025) - [Hypergraph-State Collaborative Reasoning for Multi-Object Tracking](https://mlsanity.com/paper/2604.12665/) (2026) - [IP-MOT: Instance Prompt Learning for Cross-Domain Multi-Object Tracking](https://mlsanity.com/paper/2410.23907/) (2024) - [S3MOT: Monocular 3D Object Tracking with Selective State Space Model](https://mlsanity.com/paper/2504.18068/) (2025) - [GateMOT: Q-Gated Attention for Dense Object Tracking](https://mlsanity.com/paper/2604.26353/) (2026) - [E-TraMamba: A New Paradigm for Efficient Long-Term 3D Feature Tracking with Event Cameras](https://mlsanity.com/paper/2607.02866/) (2026) ## Abstract (arXiv) Temporal motion modeling has always been a key component in multiple object tracking (MOT) which can ensure smooth trajectory movement and provide accurate positional information to enhance association precision. However, current motion models struggle to be both efficient and effective across different application scenarios. To this end, we propose TrackSSM inspired by the recently popular state space models (SSM), a unified encoder-decoder motion framework that uses data-dependent state space model to perform temporal motion of trajectories. Specifically, we propose Flow-SSM, a module that utilizes the position and motion information from historical trajectories to guide the temporal state transition of object bounding boxes. Based on Flow-SSM, we design a flow decoder. It is composed of a cascaded motion decoding module employing Flow-SSM, which can use the encoded flow information to complete the temporal position prediction of trajectories. Additionally, we propose a Step-by-Step Linear (S2L) training strategy. By performing linear interpolation between the positions of the object in the previous frame and the current frame, we construct the pseudo labels of step-by-step linear training, ensuring that the trajectory flow information can better guide the object bounding box in completing temporal transitions. TrackSSM utilizes a simple Mamba-Block to build a motion encoder for historical trajectories, forming a temporal motion model with an encoder-decoder structure in conjunction with the flow decoder. TrackSSM is applicable to various tracking scenarios and achieves excellent tracking performance across multiple benchmarks, further extending the potential of SSM-like temporal motion models in multi-object tracking tasks. Code and models are publicly available at urlhttps://github.com/Xavier-Lin/TrackSSM.