# Tracking Transforming Objects: A Benchmark

arXiv: https://arxiv.org/abs/2404.18143 · HTML: https://mlsanity.com/paper/2404.18143/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 35 of its 55 references that are in the map.
- First posted on arXiv 28 Apr 2024.

## Builds on (most cited first, 30 of 35)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Fully-Convolutional Siamese Networks for Object Tracking](https://mlsanity.com/paper/1606.09549/) (2016)
- [GOT-10k: A Large High-Diversity Benchmark for Generic Object Tracking in the Wild](https://mlsanity.com/paper/1810.11981/) (2018)
- [LaSOT: A High-quality Benchmark for Large-scale Single Object Tracking](https://mlsanity.com/paper/1809.07845/) (2018)
- [Transformer Tracking](https://mlsanity.com/paper/2103.15436/) (2021)
- [Learning Spatio-Temporal Transformer for Visual Tracking](https://mlsanity.com/paper/2103.17154/) (2021)
- [TrackingNet: A Large-Scale Dataset and Benchmark for Object Tracking in the Wild](https://mlsanity.com/paper/1803.10794/) (2018)
- [Joint Feature Learning and Relation Modeling for Tracking: A One-Stream Framework](https://mlsanity.com/paper/2203.11991/) (2022)
- [MixFormer: End-to-End Tracking with Iterative Mixed Attention](https://mlsanity.com/paper/2203.11082/) (2022)
- [Probabilistic Regression for Visual Tracking](https://mlsanity.com/paper/2003.12565/) (2020)
- [Ocean: Object-aware Anchor-free Tracking](https://mlsanity.com/paper/2006.10721/) (2020)
- [Deeper and Wider Siamese Networks for Real-Time Visual Tracking](https://mlsanity.com/paper/1901.01660/) (2019)
- [SiamCAR: Siamese Fully Convolutional Classification and Regression for Visual Tracking](https://mlsanity.com/paper/1911.07241/) (2019)
- [Need for Speed: A Benchmark for Higher Frame Rate Object Tracking](https://mlsanity.com/paper/1703.05884/) (2017)
- [LaSOT: A High-quality Large-scale Single Object Tracking Benchmark](https://mlsanity.com/paper/2009.03465/) (2020)
- [Transforming Model Prediction for Tracking](https://mlsanity.com/paper/2203.11192/) (2022)
- [A Novel Performance Evaluation Methodology for Single-Target Trackers](https://mlsanity.com/paper/1503.01313/) (2015)
- [Generalized Relation Modeling for Transformer Tracking](https://mlsanity.com/paper/2303.16580/) (2023)
- [STMTrack: Template-free Visual Tracking with Space-time Memory Networks](https://mlsanity.com/paper/2104.00324/) (2021)
- [Robust Object Modeling for Visual Tracking](https://mlsanity.com/paper/2308.05140/) (2023)
- [Autoregressive Queries for Adaptive Tracking with Spatio-TemporalTransformers](https://mlsanity.com/paper/2403.10574/) (2024)
- [Learn to Match: Automatic Matching Network Design for Visual Tracking](https://mlsanity.com/paper/2108.00803/) (2021)
- [Siamese Cascaded Region Proposal Networks for Real-Time Visual Tracking](https://mlsanity.com/paper/1812.06148/) (2018)
- [Exploring Lightweight Hierarchical Vision Transformers for Efficient Visual Tracking](https://mlsanity.com/paper/2308.06904/) (2023)
- [Explicit Visual Prompts for Visual Object Tracking](https://mlsanity.com/paper/2401.03142/) (2024)
- [Breaking the "Object" in Video Object Segmentation](https://mlsanity.com/paper/2212.06200/) (2022)
- [Robust Visual Tracking by Segmentation](https://mlsanity.com/paper/2203.11191/) (2022)
- [ZoomTrack: Target-aware Non-uniform Resizing for Efficient Visual Tracking](https://mlsanity.com/paper/2310.10071/) (2023)
- [DropMAE: Learning Representations via Masked Autoencoders with Spatial-Attention Dropout for Temporal Matching Tasks](https://mlsanity.com/paper/2304.00571/) (2023)

## Built on by (most cited first, 4 of 4)

- [Tracking and Understanding Object Transformations](https://mlsanity.com/paper/2511.04678/) (2025)
- [Tracking Reflected Objects: A Benchmark](https://mlsanity.com/paper/2407.05235/) (2024)
- [Towards Reflected Object Detection: A Benchmark](https://mlsanity.com/paper/2407.05575/) (2024)
- [Efficient Tracking and Understanding Object Transformations](https://mlsanity.com/paper/2607.19743/) (2026)

## Abstract (arXiv)

Tracking transforming objects holds significant importance in various fields due to the dynamic nature of many real-world scenarios. By enabling systems accurately represent transforming objects over time, tracking transforming objects facilitates advancements in areas such as autonomous systems, human-computer interaction, and security applications. Moreover, understanding the behavior of transforming objects provides valuable insights into complex interactions or processes, contributing to the development of intelligent systems capable of robust and adaptive perception in dynamic environments. However, current research in the field mainly focuses on tracking generic objects. In this study, we bridge this gap by collecting a novel dedicated Dataset for Tracking Transforming Objects, called DTTO, which contains 100 sequences, amounting to approximately 9.3K frames. We provide carefully hand-annotated bounding boxes for each frame within these sequences, making DTTO the pioneering benchmark dedicated to tracking transforming objects. We thoroughly evaluate 20 state-of-the-art trackers on the benchmark, aiming to comprehend the performance of existing methods and provide a comparison for future research on DTTO. With the release of DTTO, our goal is to facilitate further research and applications related to tracking transforming objects.