# SOTVerse: A User-defined Task Space of Single Object Tracking arXiv: https://arxiv.org/abs/2204.07414 ยท HTML: https://mlsanity.com/paper/2204.07414/ ## Key facts - Cited by 10 papers in the map (data through 14 Oct 2026). - Builds on 37 of its 84 references that are in the map. - First posted on arXiv 15 Apr 2022. ## Builds on (most cited first, 30 of 37) - [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014) - [Fully-Convolutional Siamese Networks for Object Tracking](https://mlsanity.com/paper/1606.09549/) (2016) - [GOT-10k: A Large High-Diversity Benchmark for Generic Object Tracking in the Wild](https://mlsanity.com/paper/1810.11981/) (2018) - [SiamRPN++: Evolution of Siamese Visual Tracking with Very Deep Networks](https://mlsanity.com/paper/1812.11703/) (2018) - [Learning Discriminative Model Prediction for Tracking](https://mlsanity.com/paper/1904.07220/) (2019) - [TrackingNet: A Large-Scale Dataset and Benchmark for Object Tracking in the Wild](https://mlsanity.com/paper/1803.10794/) (2018) - [ATOM: Accurate Tracking by Overlap Maximization](https://mlsanity.com/paper/1811.07628/) (2018) - [MixFormer: End-to-End Tracking with Iterative Mixed Attention](https://mlsanity.com/paper/2203.11082/) (2022) - [ECO: Efficient Convolution Operators for Tracking](https://mlsanity.com/paper/1611.09224/) (2016) - [Probabilistic Regression for Visual Tracking](https://mlsanity.com/paper/2003.12565/) (2020) - [Human Action Recognition and Prediction: A Survey](https://mlsanity.com/paper/1806.11230/) (2018) - [Ocean: Object-aware Anchor-free Tracking](https://mlsanity.com/paper/2006.10721/) (2020) - [Distractor-aware Siamese Networks for Visual Object Tracking](https://mlsanity.com/paper/1808.06048/) (2018) - [Open-Ended Learning Leads to Generally Capable Agents](https://mlsanity.com/paper/2107.12808/) (2021) - [SiamFC++: Towards Robust and Accurate Visual Tracking with Target Estimation Guidelines](https://mlsanity.com/paper/1911.06188/) (2019) - [Siam R-CNN: Visual Tracking by Re-Detection](https://mlsanity.com/paper/1911.12836/) (2019) - [Deeper and Wider Siamese Networks for Real-Time Visual Tracking](https://mlsanity.com/paper/1901.01660/) (2019) - [SiamCAR: Siamese Fully Convolutional Classification and Regression for Visual Tracking](https://mlsanity.com/paper/1911.07241/) (2019) - [Need for Speed: A Benchmark for Higher Frame Rate Object Tracking](https://mlsanity.com/paper/1703.05884/) (2017) - [MOTChallenge: A Benchmark for Single-Camera Multiple Target Tracking](https://mlsanity.com/paper/2010.07548/) (2020) - [LaSOT: A High-quality Large-scale Single Object Tracking Benchmark](https://mlsanity.com/paper/2009.03465/) (2020) - [Learning Target Candidate Association to Keep Track of What Not to Track](https://mlsanity.com/paper/2103.16556/) (2021) - [Augmented Reality Meets Computer Vision : Efficient Data Generation for Urban Driving Scenes](https://mlsanity.com/paper/1708.01566/) (2017) - [Know Your Surroundings: Exploiting Scene Information for Object Tracking](https://mlsanity.com/paper/2003.11014/) (2020) - [A Novel Performance Evaluation Methodology for Single-Target Trackers](https://mlsanity.com/paper/1503.01313/) (2015) - [YouTube-BoundingBoxes: A Large High-Precision Human-Annotated Data Set for Object Detection in Video](https://mlsanity.com/paper/1702.00824/) (2017) - [Deep Learning in Video Multi-Object Tracking: A Survey](https://mlsanity.com/paper/1907.12740/) (2019) - [Grounding Human-to-Vehicle Advice for Self-driving Vehicles](https://mlsanity.com/paper/1911.06978/) (2019) - [An Exploration of Embodied Visual Exploration](https://mlsanity.com/paper/2001.02192/) (2020) - [GlobalTrack: A Simple and Strong Baseline for Long-term Tracking](https://mlsanity.com/paper/1912.08531/) (2019) ## Built on by (most cited first, 10 of 10) - [DTLLM-VLT: Diverse Text Generation for Visual Language Tracking Based on LLM](https://mlsanity.com/paper/2405.12139/) (2024) - [BioDrone: A Bionic Drone-based Single Object Tracking Benchmark for Robust Vision](https://mlsanity.com/paper/2402.04519/) (2024) - [DTVLT: A Multi-modal Diverse Text Benchmark for Visual Language Tracking Based on LLM](https://mlsanity.com/paper/2410.02492/) (2024) - [Visual Language Tracking with Multi-modal Interaction: A Robust Benchmark](https://mlsanity.com/paper/2409.08887/) (2024) - [How Texts Help? A Fine-grained Evaluation to Reveal the Role of Language in Vision-Language Tracking](https://mlsanity.com/paper/2411.15600/) (2024) - [CSTrack: Enhancing RGB-X Tracking via Compact Spatiotemporal Features](https://mlsanity.com/paper/2505.19434/) (2025) - [ATCTrack: Aligning Target-Context Cues with Dynamic Target States for Robust Vision-Language Tracking](https://mlsanity.com/paper/2507.19875/) (2025) - [Enhancing Vision-Language Tracking by Effectively Converting Textual Cues into Visual Cues](https://mlsanity.com/paper/2412.19648/) (2024) - [VS-LLM: Visual-Semantic Depression Assessment based on LLM for Drawing Projection Test](https://mlsanity.com/paper/2508.05299/) (2025) - [Decoupling Amplitude and Phase Attention in Frequency Domain for RGB-Event based Visual Object Tracking](https://mlsanity.com/paper/2601.01022/) (2026) ## Abstract (arXiv) Single object tracking (SOT) research falls into a cycle -- trackers perform well on most benchmarks but quickly fail in challenging scenarios, causing researchers to doubt the insufficient data content and take more effort to construct larger datasets with more challenging situations. However, inefficient data utilization and limited evaluation methods more seriously hinder SOT research. The former causes existing datasets can not be exploited comprehensively, while the latter neglects challenging factors in the evaluation process. In this article, we systematize the representative benchmarks and form a Single Object Tracking metaverse (SOTVerse) -- a user-defined SOT task space to break through the bottleneck. We first propose a 3E Paradigm to describe tasks by three components (i.e., environment, evaluation, and executor). Then, we summarize task characteristics, clarify the organization standards, and construct SOTVerse with 12.56 million frames. Specifically, SOTVerse automatically labels challenging factors per frame, allowing users to generate user-defined spaces efficiently via construction rules. Besides, SOTVerse provides two mechanisms with new indicators and successfully evaluates trackers under various subtasks. Consequently, SOTVerse first provides a strategy to improve resource utilization in the computer vision area, making research more standardized and scientific. The SOTVerse, toolkit, evaluation server, and results are available at http://metaverse.aitestunion.com.