# Self-Supervised Multi-Object Tracking with Path Consistency

arXiv: https://arxiv.org/abs/2404.05136 · HTML: https://mlsanity.com/paper/2404.05136/

## Key facts

- Cited by 16 papers in the map (data through 14 Oct 2026).
- Builds on 45 of its 61 references that are in the map.
- First posted on arXiv 8 Apr 2024.

## Builds on (most cited first, 30 of 45)

- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [FCOS: Fully Convolutional One-Stage Object Detection](https://mlsanity.com/paper/1904.01355/) (2019)
- [Cascade R-CNN: Delving into High Quality Object Detection](https://mlsanity.com/paper/1712.00726/) (2017)
- [Objects as Points](https://mlsanity.com/paper/1904.07850/) (2019)
- [Simple Online and Realtime Tracking with a Deep Association Metric](https://mlsanity.com/paper/1703.07402/) (2017)
- [Simple Online and Realtime Tracking](https://mlsanity.com/paper/1602.00763/) (2016)
- [ByteTrack: Multi-Object Tracking by Associating Every Detection Box](https://mlsanity.com/paper/2110.06864/) (2021)
- [MOT16: A Benchmark for Multi-Object Tracking](https://mlsanity.com/paper/1603.00831/) (2016)
- [TrackFormer: Multi-Object Tracking with Transformers](https://mlsanity.com/paper/2101.02702/) (2021)
- [FairMOT: On the Fairness of Detection and Re-Identification in Multiple Object Tracking](https://mlsanity.com/paper/2004.01888/) (2020)
- [Tracking Objects as Points](https://mlsanity.com/paper/2004.01177/) (2020)
- [Tracking without bells and whistles](https://mlsanity.com/paper/1903.05625/) (2019)
- [MOTR: End-to-End Multiple-Object Tracking with Transformer](https://mlsanity.com/paper/2105.03247/) (2021)
- [MOT20: A benchmark for multi object tracking in crowded scenes](https://mlsanity.com/paper/2003.09003/) (2020)
- [Observation-Centric SORT: Rethinking SORT for Robust Multi-Object Tracking](https://mlsanity.com/paper/2203.14360/) (2022)
- [Towards Real-Time Multi-Object Tracking](https://mlsanity.com/paper/1909.12605/) (2019)
- [Quasi-Dense Similarity Learning for Multiple Object Tracking](https://mlsanity.com/paper/2006.06664/) (2020)
- [DanceTrack: Multi-Object Tracking in Uniform Appearance and Diverse Motion](https://mlsanity.com/paper/2111.14690/) (2021)
- [Probabilistic two-stage detection](https://mlsanity.com/paper/2103.07461/) (2021)
- [Learning to Track with Object Permanence](https://mlsanity.com/paper/2103.14258/) (2021)
- [Transforming Model Prediction for Tracking](https://mlsanity.com/paper/2203.11192/) (2022)
- [MeMOT: Multi-Object Tracking with Memory](https://mlsanity.com/paper/2203.16761/) (2022)
- [Global Tracking Transformers](https://mlsanity.com/paper/2203.13250/) (2022)
- [Features for Multi-Target Multi-Camera Tracking and Re-Identification](https://mlsanity.com/paper/1803.10859/) (2018)
- [MeMOTR: Long-Term Memory-Augmented Transformer for Multi-Object Tracking](https://mlsanity.com/paper/2307.15700/) (2023)
- [FAMNet: Joint Learning of Feature, Affinity and Multi-dimensional Assignment for Online Multiple Object Tracking](https://mlsanity.com/paper/1904.04989/) (2019)
- [MotionTrack: Learning Robust Short-term and Long-term Motions for Multi-Object Tracking](https://mlsanity.com/paper/2303.10404/) (2023)
- [SiamMOT: Siamese Multi-Object Tracking](https://mlsanity.com/paper/2105.11595/) (2021)
- [Robust Multi-Modality Multi-Object Tracking](https://mlsanity.com/paper/1909.03850/) (2019)

## Built on by (most cited first, 16 of 16)

- [Temporally Consistent Object-Centric Learning by Contrasting Slots](https://mlsanity.com/paper/2412.14295/) (2024)
- [Multi-objective Deep Learning: Taxonomy and Survey of the State of the Art](https://mlsanity.com/paper/2412.01566/) (2024)
- [DeCafNet: Delegate and Conquer for Efficient Temporal Grounding in Long Videos](https://mlsanity.com/paper/2505.16376/) (2025)
- [Multi-Granularity Language-Guided Training for Multi-Object Tracking](https://mlsanity.com/paper/2406.04844/) (2024)
- [Learning a Neural Association Network for Self-supervised Multi-Object Tracking](https://mlsanity.com/paper/2411.11514/) (2024)
- [STORM: End-to-End Referring Multi-Object Tracking in Videos](https://mlsanity.com/paper/2604.10527/) (2026)
- [ConsistencyTrack: A Robust Multi-Object Tracker with a Generation Strategy of Consistency Model](https://mlsanity.com/paper/2408.15548/) (2024)
- [Deep LG-Track: An Enhanced Localization-Confidence-Guided Multi-Object Tracker](https://mlsanity.com/paper/2504.01457/) (2025)
- [Self-Supervised Real-Time Tracking of Military Vehicles in Low-FPS UAV Footage](https://mlsanity.com/paper/2507.05229/) (2025)
- [Fast Self-Supervised depth and mask aware Association for Multi-Object Tracking](https://mlsanity.com/paper/2510.09878/) (2025)
- [Breaking Smooth-Motion Assumptions: A UAV Benchmark for Multi-Object Tracking in Complex and Adverse Conditions](https://mlsanity.com/paper/2603.05970/) (2026)
- [GateMOT: Q-Gated Attention for Dense Object Tracking](https://mlsanity.com/paper/2604.26353/) (2026)
- [CalibFree: Self-Supervised View Feature Separation for Calibration-Free Multi-Camera Multi-Object Tracking](https://mlsanity.com/paper/2605.09245/) (2026)
- [Cycle Consistency in Video Object-Centric Learning](https://mlsanity.com/paper/2605.30211/) (2026)
- [PS-MOT: Cultivating Instance Awareness from Point Seeds for Multi-Object Tracking](https://mlsanity.com/paper/2606.30476/) (2026)
- [Cross-Domain Tracker Adaptation Without Target-Domain Labels via Vision-Language Agents](https://mlsanity.com/paper/2609.05239/) (2026)

## Abstract (arXiv)

In this paper, we propose a novel concept of path consistency to learn robust object matching without using manual object identity supervision. Our key idea is that, to track a object through frames, we can obtain multiple different association results from a model by varying the frames it can observe, i.e., skipping frames in observation. As the differences in observations do not alter the identities of objects, the obtained association results should be consistent. Based on this rationale, we generate multiple observation paths, each specifying a different set of frames to be skipped, and formulate the Path Consistency Loss that enforces the association results are consistent across different observation paths. We use the proposed loss to train our object matching model with only self-supervision. By extensive experiments on three tracking datasets (MOT17, PersonPath22, KITTI), we demonstrate that our method outperforms existing unsupervised methods with consistent margins on various evaluation metrics, and even achieves performance close to supervised methods.