mlsanityData to 7 Oct 2026

Contrastive Learning for Multi-Object Tracking with Transformers

arXiv 2311.08043 ↗First posted 14 Nov 2023cs.CVcited by 6

Not on the map. Why?

Abstract (arXiv)

The DEtection TRansformer (DETR) opened new possibilities for object detection by modeling it as a translation task: converting image features into object-level representations. Previous works typically add expensive modules to DETR to perform Multi-Object Tracking (MOT), resulting in more complicated architectures. We instead show how DETR can be turned into a MOT model by employing an instance-level contrastive loss, a revised sampling strategy and a lightweight assignment method. Our training scheme learns object appearances while preserving detection capabilities and with little overhead. Its performance surpasses the previous state-of-the-art by +2.6 mMOTA on the challenging BDD100K dataset and is comparable to existing transformer-based methods on the MOT17 dataset.

Key facts. Cited by 6 papers in the map (data through 7 Oct 2026). Builds on 44 of its 52 references that are in the map. First posted on arXiv 14 Nov 2023.

Builds on

44 of 52 references.

References · 44

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  4. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  5. 2021Swin Transformer: Hierarchical Vision Transformer using Shifted Windowscited by 12,082
  6. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  7. 2015Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networkscited by 9,617
  8. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  9. 2020End-to-End Object Detection with Transformerscited by 7,384
  10. 2015You Only Look Once: Unified, Real-Time Object Detectioncited by 4,829
  11. 2020Deformable DETR: Deformable Transformers for End-to-End Object Detectioncited by 3,410
  12. 2020Supervised Contrastive Learningcited by 2,803
32 more
  1. 2013Rich feature hierarchies for accurate object detection and semantic segmentationcited by 2,160
  2. 2022DINO: DETR with Improved DeNoising Anchor Boxes for End-to-End Object Detectioncited by 1,306
  3. 2018BDD100K: A Diverse Driving Dataset for Heterogeneous Multitask Learningcited by 1,282
  4. 2018Learning deep representations by mutual information estimation and maximizationcited by 1,265
  5. 2021YOLOX: Exceeding YOLO Series in 2021cited by 1,064
  6. 2021ByteTrack: Multi-Object Tracking by Associating Every Detection Boxcited by 812
  7. 2016Performance Measures and a Data Set for Multi-Target, Multi-Camera Trackingcited by 685
  8. 2019Data-Efficient Image Recognition with Contrastive Predictive Codingcited by 672
  9. 2020HOTA: A Higher Order Metric for Evaluating Multi-Object Trackingcited by 546
  10. 2016MOT16: A Benchmark for Multi-Object Trackingcited by 531
  11. 2021TrackFormer: Multi-Object Tracking with Transformerscited by 515
  12. 2020FairMOT: On the Fairness of Detection and Re-Identification in Multiple Object Trackingcited by 489
  13. 2020Tracking Objects as Pointscited by 483
  14. 2021MOTR: End-to-End Multiple-Object Tracking with Transformercited by 379
  15. 2020TransTrack: Multiple Object Tracking with Transformercited by 370
  16. 2019Towards Real-Time Multi-Object Trackingcited by 311
  17. 2018CrowdHuman: A Benchmark for Detecting Human in a Crowdcited by 283
  18. 2020Quasi-Dense Similarity Learning for Multiple Object Trackingcited by 220
  19. 2016Joint Detection and Identification Feature Learning for Person Searchcited by 190
  20. 2017CityPersons: A Diverse Dataset for Pedestrian Detectioncited by 171
  21. 2016Person Re-identification in the Wildcited by 157
  22. 2022MOTRv2: Bootstrapping End-to-End Multi-Object Tracking by Pretrained Object Detectorscited by 148
  23. 2017Detect to Track and Track to Detectcited by 131
  24. 2021Learning to Track with Object Permanencecited by 117
  25. 2022MeMOT: Multi-Object Tracking with Memorycited by 113
  26. 2020RetinaTrack: Online Single Stage Joint Detection and Trackingcited by 67
  27. 2022Unifying Short and Long-Term Tracking with Graph Hierarchiescited by 62
  28. 2022Tracking Every Thing in the Wildcited by 44
  29. 2021TransCenter: Transformers with Dense Representations for Multiple-Object Trackingcited by 42
  30. 2022Towards Discriminative Representation: Multi-view Trajectory Contrastive Learning for Online Multi-object Trackingcited by 33
  31. 2021Looking Beyond Two Frames: End-to-End Multi-Object Tracking Using Spatial and Temporal Transformerscited by 21
  32. 2022End-to-end Tracking with a Multi-query Transformercited by 5

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 2; 2025: 2; 2026: 2.

6 papers.

  1. 2024Multiple Object Tracking as ID Predictioncited by 58
  2. 2024Multi-Granularity Language-Guided Training for Multi-Object Trackingcited by 2
  3. 2025History-Aware Transformation of ReID Features for Multiple Object Trackingcited by 1
  4. 2025MANTA: Physics-Informed Generalized Underwater Object Trackingcited by 0
  5. 2026Self-Supervised Animal Identification for Long Videoscited by 0
  6. 2026Beyond Pedestrians: Caption-Guided CLIP Framework for High-Difficulty Video-based Person Re-Identificationcited by 0