mlsanityData to 7 Oct 2026

ByteTrack: Multi-Object Tracking by Associating Every Detection Box

arXiv 2110.06864 ↗First posted 13 Oct 2021cs.CVcited by 812

Not on the map. Why?

Abstract (arXiv)

Multi-object tracking (MOT) aims at estimating bounding boxes and identities of objects in videos. Most methods obtain identities by associating detection boxes whose scores are higher than a threshold. The objects with low detection scores, e.g. occluded objects, are simply thrown away, which brings non-negligible true object missing and fragmented trajectories. To solve this problem, we present a simple, effective and generic association method, tracking by associating almost every detection box instead of only the high score ones. For the low score detection boxes, we utilize their similarities with tracklets to recover true objects and filter out the background detections. When applied to 9 different state-of-the-art trackers, our method achieves consistent improvement on IDF1 score ranging from 1 to 10 points. To put forwards the state-of-the-art performance of MOT, we design a simple and strong tracker, named ByteTrack. For the first time, we achieve 80.3 MOTA, 77.3 IDF1 and 63.1 HOTA on the test set of MOT17 with 30 FPS running speed on a single V100 GPU. ByteTrack also achieves state-of-the-art performance on MOT20, HiEve and BDD100K tracking benchmarks. The source code, pre-trained models with deploy versions and tutorials of applying to other trackers are released at https://github.com/ifzhang/ByteTrack.

Key facts. Cited by 812 papers in the map (data through 7 Oct 2026). Builds on 75 of its 104 references that are in the map. First posted on arXiv 13 Oct 2021.

Builds on

75 of 104 references.

References · 75

  1. 2017Attention Is All You Needcited by 59,932
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  3. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  4. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  5. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,617
  6. 2020End-to-End Object Detection with Transformerscited by 7,384
  7. 2017Mask R-CNNcited by 7,259
  8. 2017mixup: Beyond Empirical Risk Minimizationcited by 5,155
  9. 2020Deformable DETR: Deformable Transformers for End-to-End Object Detectioncited by 3,410
  10. 2018YOLOv3: An Incremental Improvementcited by 2,652
  11. 2021Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction without Convolutionscited by 1,960
  12. 2020YOLOv4: Optimal Speed and Accuracy of Object Detectioncited by 1,624
38 more
  1. 2019Generalized Intersection over Union: A Metric and A Loss for Bounding Box Regressioncited by 1,361
  2. 2018BDD100K: A Diverse Driving Dataset for Heterogeneous Multitask Learningcited by 1,282
  3. 2017Cascade R-CNN: Delving into High Quality Object Detectioncited by 1,275
  4. 2019Objects as Pointscited by 1,172
  5. 2017Mixed Precision Trainingcited by 1,135
  6. 2021YOLOX: Exceeding YOLO Series in 2021cited by 1,064
  7. 2017Simple Online and Realtime Tracking with a Deep Association Metriccited by 908
  8. 2016Simple Online and Realtime Trackingcited by 862
  9. 2018Path Aggregation Network for Instance Segmentationcited by 796
  10. 2016Performance Measures and a Data Set for Multi-Target, Multi-Camera Trackingcited by 685
  11. 2016Fully-Convolutional Siamese Networks for Object Trackingcited by 566
  12. 2020HOTA: A Higher Order Metric for Evaluating Multi-Object Trackingcited by 546
  13. 2016MOT16: A Benchmark for Multi-Object Trackingcited by 531
  14. 2021TrackFormer: Multi-Object Tracking with Transformerscited by 515
  15. 2020Sparse R-CNN: End-to-End Object Detection with Learnable Proposalscited by 494
  16. 2020FairMOT: On the Fairness of Detection and Re-Identification in Multiple Object Trackingcited by 489
  17. 2020Tracking Objects as Pointscited by 483
  18. 2019Tracking without bells and whistlescited by 382
  19. 2019CSPNet: A New Backbone that can Enhance Learning Capability of CNNcited by 381
  20. 2021MOTR: End-to-End Multiple-Object Tracking with Transformercited by 379
  21. 2020TransTrack: Multiple Object Tracking with Transformercited by 370
  22. 2020MOT20: A benchmark for multi object tracking in crowded scenescited by 355
  23. 2019Towards Real-Time Multi-Object Trackingcited by 311
  24. 2018CrowdHuman: A Benchmark for Detecting Human in a Crowdcited by 283
  25. 2020Quasi-Dense Similarity Learning for Multiple Object Trackingcited by 220
  26. 2019Learning a Neural Solver for Multiple Object Trackingcited by 198
  27. 2016Joint Detection and Identification Feature Learning for Person Searchcited by 190
  28. 2021OTA: Optimal Transport Assignment for Object Detectioncited by 177
  29. 2017CityPersons: A Diverse Dataset for Pedestrian Detectioncited by 171
  30. 2021You Only Look at One Sequence: Rethinking Transformer in Vision through Object Detectioncited by 170
  31. 2020FastReID: A Pytorch Toolbox for General Instance Re-identificationcited by 164
  32. 2016Person Re-identification in the Wildcited by 157
  33. 2021Track to Detect and Segment: An Online Multi-Object Trackercited by 156
  34. 2021Learning to Track with Object Permanencecited by 117
  35. 2020Chained-Tracker: Chaining Paired Attentive Regression Results for End-to-End Joint Multiple-Object Detection and Trackingcited by 115
  36. 2020Rethinking the competition between detection and ReID in Multi-Object Trackingcited by 101
  37. 2020AutoAssign: Differentiable Label Assignment for Dense Object Detectioncited by 100
  38. 2018Real-time Multiple People Tracking with Deeply Learned Candidate Selection and Person Re-Identificationcited by 96

50 most cited of 75 All 75 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 3; 2022: 84; 2023: 161; 2024: 187; 2025: 198; 2026: 179.

812 papers.

Sort:

By area:

  1. 2022YOLOv7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectorscited by 978
  2. 2024Grounded SAM: Assembling Open-World Models for Diverse Visual Taskscited by 914
  3. 2023DETRs Beat YOLOs on Real-time Object Detectioncited by 526
  4. 2022Observation-Centric SORT: Rethinking SORT for Robust Multi-Object Trackingcited by 345
  5. 2024DiffusionDrive: Truncated Diffusion Model for End-to-End Autonomous Drivingcited by 324
  6. 2022BoT-SORT: Robust Associations Multi-Pedestrian Trackingcited by 313
  7. 2024VideoAgent: A Memory-augmented Multimodal Agent for Video Understandingcited by 224
  8. 2021DanceTrack: Multi-Object Tracking in Uniform Appearance and Diverse Motioncited by 213
  9. 2022StrongSORT: Make DeepSORT Great Againcited by 213
  10. 2023Universal Instance Perception as Object Discovery and Retrievalcited by 190
  11. 2024Identity-Preserving Text-to-Video Generation by Frequency Decompositioncited by 155
  12. 2022MOTRv2: Bootstrapping End-to-End Multi-Object Tracking by Pretrained Object Detectorscited by 148
38 more
  1. 2022AiATrack: Attention in Attention for Transformer Visual Trackingcited by 137
  2. 2023Deep OC-SORT: Multi-Pedestrian Tracking by Adaptive Re-Identificationcited by 130
  3. 2023SportsMOT: A Large Multi-Object Tracking Dataset in Multiple Sports Scenescited by 130
  4. 2021SimpleTrack: Understanding and Rethinking 3D Multi-object Trackingcited by 102
  5. 2022In Defense of Online Models for Video Instance Segmentationcited by 97
  6. 2022Towards Grand Unification of Object Trackingcited by 93
  7. 2023ReasonNet: End-to-End Driving with Temporal and Global Reasoningcited by 92
  8. 2023Referring Multi-Object Trackingcited by 91
  9. 2024SAMURAI: Adapting Segment Anything Model for Zero-Shot Visual Tracking with Motion-Aware Memorycited by 91
  10. 2023TRACE: 5D Temporal Regression of Avatars with Dynamic Cameras in 3D Environmentscited by 89
  11. 2023MeMOTR: Long-Term Memory-Augmented Transformer for Multi-Object Trackingcited by 86
  12. 2022SoccerNet-Tracking: Multiple Object Tracking Dataset and Benchmark in Soccer Videoscited by 80
  13. 2023General Object Foundation Model for Images and Videos at Scalecited by 71
  14. 2022Hard to Track Objects with Irregular Motions and Similar Appearances? Make It Easier by Buffering the Matching Spacecited by 66
  15. 2023Hybrid-SORT: Weak Cues Matter for Online Multi-Object Trackingcited by 66
  16. 2023MotionTrack: Learning Robust Short-term and Long-term Motions for Multi-Object Trackingcited by 64
  17. 2023A Comprehensive Survey on Segment Anything Model for Vision and Beyondcited by 64
  18. 2022Unifying Short and Long-Term Tracking with Graph Hierarchiescited by 62
  19. 2022QDTrack: Quasi-Dense Similarity Learning for Appearance-Only Multiple Object Trackingcited by 61
  20. 2022Expanding Accurate Person Recognition to New Altitudes and Ranges: The BRIAR Datasetcited by 61
  21. 2024DiffMOT: A Real-time Diffusion-based Multiple Object Tracker with Non-linear Predictioncited by 59
  22. 2023EgoHumans: An Egocentric 3D Multi-Human Benchmarkcited by 58
  23. 2024Multiple Object Tracking as ID Predictioncited by 58
  24. 2022APT-36K: A Large-scale Benchmark for Animal Pose Estimation and Trackingcited by 55
  25. 2022LidarGait: Benchmarking 3D Gait Recognition with Point Cloudscited by 54
  26. 2022GaitEdge: Beyond Plain End-to-end Gait Recognition for Better Practicalitycited by 52
  27. 2025DiffusionDriveV2: Reinforcement Learning-Constrained Truncated Diffusion Modeling in End-to-End Autonomous Drivingcited by 52
  28. 2023PointOcc: Cylindrical Tri-Perspective View for Point-based 3D Semantic Occupancy Predictioncited by 51
  29. 2022CAMO-MOT: Combined Appearance-Motion Optimization for 3D Multi-Object Tracking with Camera-LiDAR Fusioncited by 50
  30. 2024Matching Anything by Segmenting Anythingcited by 49
  31. 2023SparseTrack: Multi-Object Tracking by Performing Scene Decomposition based on Pseudo-Depthcited by 48
  32. 2022Simple Cues Lead to a Strong Multi-Object Trackercited by 44
  33. 2023A New Comprehensive Benchmark for Semi-supervised Video Anomaly Detection and Anticipationcited by 42
  34. 2023MOTRv3: Release-Fetch Supervision for End-to-End Multi-Object Trackingcited by 42
  35. 2023UCMCTrack: Multi-Object Tracking with Uniform Camera Motion Compensationcited by 42
  36. 2023DiffusionTrack: Diffusion Model For Multi-Object Trackingcited by 40
  37. 2022AnimalTrack: A Benchmark for Multi-Animal Tracking in the Wildcited by 37
  38. 2023MAS: Multi-view Ancestral Sampling for 3D motion generation using 2D diffusioncited by 37

50 most cited of 812 All 812 (JSON)