# DanceTrack: Multi-Object Tracking in Uniform Appearance and Diverse Motion

arXiv: https://arxiv.org/abs/2111.14690 · HTML: https://mlsanity.com/paper/2111.14690/

## Key facts

- Cited by 217 papers in the map (data through 14 Oct 2026).
- Builds on 36 of its 56 references that are in the map.
- First posted on arXiv 29 Nov 2021.

## Builds on (most cited first, 30 of 36)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Scalability in Perception for Autonomous Driving: Waymo Open Dataset](https://mlsanity.com/paper/1912.04838/) (2019)
- [Objects as Points](https://mlsanity.com/paper/1904.07850/) (2019)
- [YOLOX: Exceeding YOLO Series in 2021](https://mlsanity.com/paper/2107.08430/) (2021)
- [Simple Online and Realtime Tracking with a Deep Association Metric](https://mlsanity.com/paper/1703.07402/) (2017)
- [Simple Online and Realtime Tracking](https://mlsanity.com/paper/1602.00763/) (2016)
- [ByteTrack: Multi-Object Tracking by Associating Every Detection Box](https://mlsanity.com/paper/2110.06864/) (2021)
- [Performance Measures and a Data Set for Multi-Target, Multi-Camera Tracking](https://mlsanity.com/paper/1609.01775/) (2016)
- [KITTI-360: A Novel Dataset and Benchmarks for Urban Scene Understanding in 2D and 3D](https://mlsanity.com/paper/2109.13410/) (2021)
- [HOTA: A Higher Order Metric for Evaluating Multi-Object Tracking](https://mlsanity.com/paper/2009.07736/) (2020)
- [MOT16: A Benchmark for Multi-Object Tracking](https://mlsanity.com/paper/1603.00831/) (2016)
- [TrackFormer: Multi-Object Tracking with Transformers](https://mlsanity.com/paper/2101.02702/) (2021)
- [FairMOT: On the Fairness of Detection and Re-Identification in Multiple Object Tracking](https://mlsanity.com/paper/2004.01888/) (2020)
- [Tracking Objects as Points](https://mlsanity.com/paper/2004.01177/) (2020)
- [YouTube-VOS: A Large-Scale Video Object Segmentation Benchmark](https://mlsanity.com/paper/1809.03327/) (2018)
- [Tracking without bells and whistles](https://mlsanity.com/paper/1903.05625/) (2019)
- [MOTR: End-to-End Multiple-Object Tracking with Transformer](https://mlsanity.com/paper/2105.03247/) (2021)
- [TransTrack: Multiple Object Tracking with Transformer](https://mlsanity.com/paper/2012.15460/) (2020)
- [MOT20: A benchmark for multi object tracking in crowded scenes](https://mlsanity.com/paper/2003.09003/) (2020)
- [Towards Real-Time Multi-Object Tracking](https://mlsanity.com/paper/1909.12605/) (2019)
- [Bidirectional Learning for Domain Adaptation of Semantic Segmentation](https://mlsanity.com/paper/1904.10620/) (2019)
- [MOTS: Multi-Object Tracking and Segmentation](https://mlsanity.com/paper/1902.03604/) (2019)
- [Quasi-Dense Similarity Learning for Multiple Object Tracking](https://mlsanity.com/paper/2006.06664/) (2020)
- [Track to Detect and Segment: An Online Multi-Object Tracker](https://mlsanity.com/paper/2103.08808/) (2021)
- [BoxInst: High-Performance Instance Segmentation with Box Annotations](https://mlsanity.com/paper/2012.02310/) (2020)
- [TAO: A Large-Scale Benchmark for Tracking Any Object](https://mlsanity.com/paper/2005.10356/) (2020)
- [MOTChallenge 2015: Towards a Benchmark for Multi-Target Tracking](https://mlsanity.com/paper/1504.01942/) (2015)
- [Towards 3D Human Pose Estimation in the Wild: a Weakly-supervised Approach](https://mlsanity.com/paper/1704.02447/) (2017)
- [Cross-Domain Adaptation for Animal Pose Estimation](https://mlsanity.com/paper/1908.05806/) (2019)

## Built on by (most cited first, 30 of 217)

- [Observation-Centric SORT: Rethinking SORT for Robust Multi-Object Tracking](https://mlsanity.com/paper/2203.14360/) (2022)
- [StrongSORT: Make DeepSORT Great Again](https://mlsanity.com/paper/2202.13514/) (2022)
- [Video Understanding with Large Language Models: A Survey](https://mlsanity.com/paper/2312.17432/) (2023)
- [MOTRv2: Bootstrapping End-to-End Multi-Object Tracking by Pretrained Object Detectors](https://mlsanity.com/paper/2211.09791/) (2022)
- [Deep OC-SORT: Multi-Pedestrian Tracking by Adaptive Re-Identification](https://mlsanity.com/paper/2302.11813/) (2023)
- [SportsMOT: A Large Multi-Object Tracking Dataset in Multiple Sports Scenes](https://mlsanity.com/paper/2304.05170/) (2023)
- [MeMOTR: Long-Term Memory-Augmented Transformer for Multi-Object Tracking](https://mlsanity.com/paper/2307.15700/) (2023)
- [Hybrid-SORT: Weak Cues Matter for Online Multi-Object Tracking](https://mlsanity.com/paper/2308.00783/) (2023)
- [Hard to Track Objects with Irregular Motions and Similar Appearances? Make It Easier by Buffering the Matching Space](https://mlsanity.com/paper/2211.14317/) (2022)
- [Unifying Short and Long-Term Tracking with Graph Hierarchies](https://mlsanity.com/paper/2212.03038/) (2022)
- [QDTrack: Quasi-Dense Similarity Learning for Appearance-Only Multiple Object Tracking](https://mlsanity.com/paper/2210.06984/) (2022)
- [EgoHumans: An Egocentric 3D Multi-Human Benchmark](https://mlsanity.com/paper/2305.16487/) (2023)
- [Multiple Object Tracking as ID Prediction](https://mlsanity.com/paper/2403.16848/) (2024)
- [DiffMOT: A Real-time Diffusion-based Multiple Object Tracker with Non-linear Prediction](https://mlsanity.com/paper/2403.02075/) (2024)
- [Vitron: A Unified Pixel-level Vision LLM for Understanding, Generating, Segmenting, Editing](https://mlsanity.com/paper/2412.19806/) (2024)
- [SparseTrack: Multi-Object Tracking by Performing Scene Decomposition based on Pseudo-Depth](https://mlsanity.com/paper/2306.05238/) (2023)
- [Merlin:Empowering Multimodal LLMs with Foresight Minds](https://mlsanity.com/paper/2312.00589/) (2023)
- [TAPTR: Tracking Any Point with Transformers as Detection](https://mlsanity.com/paper/2403.13042/) (2024)
- [Simple Cues Lead to a Strong Multi-Object Tracker](https://mlsanity.com/paper/2206.04656/) (2022)
- [UCMCTrack: Multi-Object Tracking with Uniform Camera Motion Compensation](https://mlsanity.com/paper/2312.08952/) (2023)
- [LVOS: A Benchmark for Large-scale Long-term Video Object Segmentation](https://mlsanity.com/paper/2404.19326/) (2024)
- [MOTRv3: Release-Fetch Supervision for End-to-End Multi-Object Tracking](https://mlsanity.com/paper/2305.14298/) (2023)
- [DiffusionTrack: Diffusion Model For Multi-Object Tracking](https://mlsanity.com/paper/2308.09905/) (2023)
- [AnimalTrack: A Benchmark for Multi-Animal Tracking in the Wild](https://mlsanity.com/paper/2205.00158/) (2022)
- [Music-Driven Group Choreography](https://mlsanity.com/paper/2303.12337/) (2023)
- [iKUN: Speak to Trackers without Retraining](https://mlsanity.com/paper/2312.16245/) (2023)
- [Iterative Scale-Up ExpansionIoU and Deep Features Association for Multi-Object Tracking in Sports](https://mlsanity.com/paper/2306.13074/) (2023)
- [Focus On Details: Online Multi-object Tracking with Diverse Fine-grained Representation](https://mlsanity.com/paper/2302.14589/) (2023)
- [Towards Generalizable Multi-Object Tracking](https://mlsanity.com/paper/2406.00429/) (2024)
- [DVIS-DAQ: Improving Video Segmentation via Dynamic Anchor Queries](https://mlsanity.com/paper/2404.00086/) (2024)

## Abstract (arXiv)

A typical pipeline for multi-object tracking (MOT) is to use a detector for object localization, and following re-identification (re-ID) for object association. This pipeline is partially motivated by recent progress in both object detection and re-ID, and partially motivated by biases in existing tracking datasets, where most objects tend to have distinguishing appearance and re-ID models are sufficient for establishing associations. In response to such bias, we would like to re-emphasize that methods for multi-object tracking should also work when object appearance is not sufficiently discriminative. To this end, we propose a large-scale dataset for multi-human tracking, where humans have similar appearance, diverse motion and extreme articulation. As the dataset contains mostly group dancing videos, we name it "DanceTrack". We expect DanceTrack to provide a better platform to develop more MOT algorithms that rely less on visual discrimination and depend more on motion analysis. We benchmark several state-of-the-art trackers on our dataset and observe a significant performance drop on DanceTrack when compared against existing benchmarks. The dataset, project code and competition server are released at: urlhttps://github.com/DanceTrack.