# TransCenter: Transformers with Dense Representations for Multiple-Object Tracking

arXiv: https://arxiv.org/abs/2103.15145 · HTML: https://mlsanity.com/paper/2103.15145/

## Key facts

- Cited by 42 papers in the map (data through 14 Oct 2026).
- Builds on 55 of its 96 references that are in the map.
- First posted on arXiv 28 Mar 2021.

## Builds on (most cited first, 30 of 55)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [End-to-End Object Detection with Transformers](https://mlsanity.com/paper/2005.12872/) (2020)
- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [You Only Look Once: Unified, Real-Time Object Detection](https://mlsanity.com/paper/1506.02640/) (2015)
- [Deformable DETR: Deformable Transformers for End-to-End Object Detection](https://mlsanity.com/paper/2010.04159/) (2020)
- [Deformable Convolutional Networks](https://mlsanity.com/paper/1703.06211/) (2017)
- [Objects as Points](https://mlsanity.com/paper/1904.07850/) (2019)
- [Simple Online and Realtime Tracking](https://mlsanity.com/paper/1602.00763/) (2016)
- [CornerNet: Detecting Objects as Paired Keypoints](https://mlsanity.com/paper/1808.01244/) (2018)
- [Performance Measures and a Data Set for Multi-Target, Multi-Camera Tracking](https://mlsanity.com/paper/1609.01775/) (2016)
- [MOT16: A Benchmark for Multi-Object Tracking](https://mlsanity.com/paper/1603.00831/) (2016)
- [TrackFormer: Multi-Object Tracking with Transformers](https://mlsanity.com/paper/2101.02702/) (2021)
- [FairMOT: On the Fairness of Detection and Re-Identification in Multiple Object Tracking](https://mlsanity.com/paper/2004.01888/) (2020)
- [Tracking Objects as Points](https://mlsanity.com/paper/2004.01177/) (2020)
- [TransReID: Transformer-based Object Re-Identification](https://mlsanity.com/paper/2102.04378/) (2021)
- [Tracking without bells and whistles](https://mlsanity.com/paper/1903.05625/) (2019)
- [TransTrack: Multiple Object Tracking with Transformer](https://mlsanity.com/paper/2012.15460/) (2020)
- [MOT20: A benchmark for multi object tracking in crowded scenes](https://mlsanity.com/paper/2003.09003/) (2020)
- [Learning Texture Transformer Network for Image Super-Resolution](https://mlsanity.com/paper/2006.04139/) (2020)
- [CrowdHuman: A Benchmark for Detecting Human in a Crowd](https://mlsanity.com/paper/1805.00123/) (2018)
- [Quasi-Dense Similarity Learning for Multiple Object Tracking](https://mlsanity.com/paper/2006.06664/) (2020)
- [Learning a Neural Solver for Multiple Object Tracking](https://mlsanity.com/paper/1912.07515/) (2019)
- [CityPersons: A Diverse Dataset for Pedestrian Detection](https://mlsanity.com/paper/1702.05693/) (2017)
- [Person Re-identification in the Wild](https://mlsanity.com/paper/1604.02531/) (2016)
- [Track to Detect and Segment: An Online Multi-Object Tracker](https://mlsanity.com/paper/2103.08808/) (2021)
- [Learning to Track with Object Permanence](https://mlsanity.com/paper/2103.14258/) (2021)

## Built on by (most cited first, 30 of 42)

- [SparseTrack: Multi-Object Tracking by Performing Scene Decomposition based on Pseudo-Depth](https://mlsanity.com/paper/2306.05238/) (2023)
- [DiffusionTrack: Diffusion Model For Multi-Object Tracking](https://mlsanity.com/paper/2308.09905/) (2023)
- [ADA-Track++: End-to-End Multi-Camera 3D Multi-Object Tracking with Alternating Detection and Association](https://mlsanity.com/paper/2405.08909/) (2024)
- [SMILEtrack: SiMIlarity LEarning for Occlusion-Aware Multiple Object Tracking](https://mlsanity.com/paper/2211.08824/) (2022)
- [NetTrack: Tracking Highly Dynamic Objects with a Net](https://mlsanity.com/paper/2403.11186/) (2024)
- [Samba: Synchronized Set-of-Sequences Modeling for Multiple Object Tracking](https://mlsanity.com/paper/2410.01806/) (2024)
- [MotionTrack: Learning Motion Predictor for Multiple Object Tracking](https://mlsanity.com/paper/2306.02585/) (2023)
- [MotionTrack: End-to-End Transformer-based Multi-Object Tracing with LiDAR-Camera Fusion](https://mlsanity.com/paper/2306.17000/) (2023)
- [Generative Transformer for Accurate and Reliable Salient Object Detection](https://mlsanity.com/paper/2104.10127/) (2021)
- [TAPTRv2: Attention-based Position Update Improves Tracking Any Point](https://mlsanity.com/paper/2407.16291/) (2024)
- [VcT: Visual change Transformer for Remote Sensing Image Change Detection](https://mlsanity.com/paper/2310.11417/) (2023)
- [ETTrack: Enhanced Temporal Motion Predictor for Multi-Object Tracking](https://mlsanity.com/paper/2405.15755/) (2024)
- [Radar and Camera Fusion for Object Detection and Tracking: A Comprehensive Survey](https://mlsanity.com/paper/2410.19872/) (2024)
- [Contrastive Learning for Multi-Object Tracking with Transformers](https://mlsanity.com/paper/2311.08043/) (2023)
- [Valeo4Cast: A Modular Approach to End-to-End Forecasting](https://mlsanity.com/paper/2406.08113/) (2024)
- [MVTrajecter: Multi-View Pedestrian Tracking with Trajectory Motion Cost and Trajectory Appearance Cost](https://mlsanity.com/paper/2509.01157/) (2025)
- [TQD-Track: Temporal Query Denoising for 3D Multi-Object Tracking](https://mlsanity.com/paper/2504.03258/) (2025)
- [Multi-Object Tracking as Attention Mechanism](https://mlsanity.com/paper/2307.05874/) (2023)
- [LDTrack: Dynamic People Tracking by Service Robots using Diffusion Models](https://mlsanity.com/paper/2402.08774/) (2024)
- [TGOSPA Metric Parameters Selection and Evaluation for Visual Multi-object Tracking](https://mlsanity.com/paper/2412.08321/) (2024)
- [When Trackers Date Fish: A Benchmark and Framework for Underwater Multiple Fish Tracking](https://mlsanity.com/paper/2507.06400/) (2025)
- [Hypergraph-State Collaborative Reasoning for Multi-Object Tracking](https://mlsanity.com/paper/2604.12665/) (2026)
- [Transformer-based assignment decision network for multiple object tracking](https://mlsanity.com/paper/2208.03571/) (2022)
- [Strong-TransCenter: Improved Multi-Object Tracking based on Transformers with Dense Representations](https://mlsanity.com/paper/2210.13570/) (2022)
- [Socially Pertinent Robots in Gerontological Healthcare](https://mlsanity.com/paper/2404.07560/) (2024)
- [FMRFT: Fusion Mamba and DETR for Query Time Sequence Intersection Fish Tracking](https://mlsanity.com/paper/2409.01148/) (2024)
- [IP-MOT: Instance Prompt Learning for Cross-Domain Multi-Object Tracking](https://mlsanity.com/paper/2410.23907/) (2024)
- [FastTrackTr:Towards Fast Multi-Object Tracking with Transformers](https://mlsanity.com/paper/2411.15811/) (2024)
- [Heterogeneous Graph Transformer for Multiple Tiny Object Tracking in RGB-T Videos](https://mlsanity.com/paper/2412.10861/) (2024)
- [Depth-Aware Scoring and Hierarchical Alignment for Multiple Object Tracking](https://mlsanity.com/paper/2506.00774/) (2025)

## Abstract (arXiv)

Transformers have proven superior performance for a wide variety of tasks since they were introduced. In recent years, they have drawn attention from the vision community in tasks such as image classification and object detection. Despite this wave, an accurate and efficient multiple-object tracking (MOT) method based on transformers is yet to be designed. We argue that the direct application of a transformer architecture with quadratic complexity and insufficient noise-initialized sparse queries - is not optimal for MOT. We propose TransCenter, a transformer-based MOT architecture with dense representations for accurately tracking all the objects while keeping a reasonable runtime. Methodologically, we propose the use of image-related dense detection queries and efficient sparse tracking queries produced by our carefully designed query learning networks (QLN). On one hand, the dense image-related detection queries allow us to infer targets' locations globally and robustly through dense heatmap outputs. On the other hand, the set of sparse tracking queries efficiently interacts with image features in our TransCenter Decoder to associate object positions through time. As a result, TransCenter exhibits remarkable performance improvements and outperforms by a large margin the current state-of-the-art methods in two standard MOT benchmarks with two tracking settings (public/private). TransCenter is also proven efficient and accurate by an extensive ablation study and comparisons to more naive alternatives and concurrent works. For scientific interest, the code is made publicly available at https://github.com/yihongxu/transcenter.