# Describe and Attend to Track: Learning Natural Language guided Structural Representation and Visual Attention for Object Tracking

arXiv: https://arxiv.org/abs/1811.10014 · HTML: https://mlsanity.com/paper/1811.10014/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 Nov 2018.

## Built on by (most cited first, 20 of 20)

- [Large-scale Multi-Modal Pre-trained Models: A Comprehensive Survey](https://mlsanity.com/paper/2302.10035/) (2023)
- [Towards More Flexible and Accurate Object Tracking with Natural Language: Algorithms and Benchmark](https://mlsanity.com/paper/2103.16746/) (2021)
- [Towards Unified Token Learning for Vision-Language Tracking](https://mlsanity.com/paper/2308.14103/) (2023)
- [Joint Visual Grounding and Tracking with Natural Language Specification](https://mlsanity.com/paper/2303.12027/) (2023)
- [SUTrack: Towards Simple and Unified Single Object Tracking](https://mlsanity.com/paper/2412.19138/) (2024)
- [MFGNet: Dynamic Modality-Aware Filter Generation for RGB-T Tracking](https://mlsanity.com/paper/2107.10433/) (2021)
- [DTLLM-VLT: Diverse Text Generation for Visual Language Tracking Based on LLM](https://mlsanity.com/paper/2405.12139/) (2024)
- [Unified Sequence-to-Sequence Learning for Single- and Multi-Modal Visual Object Tracking](https://mlsanity.com/paper/2304.14394/) (2023)
- [Dynamic Attention guided Multi-Trajectory Analysis for Single Object Tracking](https://mlsanity.com/paper/2103.16086/) (2021)
- [DTVLT: A Multi-modal Diverse Text Benchmark for Visual Language Tracking Based on LLM](https://mlsanity.com/paper/2410.02492/) (2024)
- [How Texts Help? A Fine-grained Evaluation to Reveal the Role of Language in Vision-Language Tracking](https://mlsanity.com/paper/2411.15600/) (2024)
- [ATCTrack: Aligning Target-Context Cues with Dynamic Target States for Robust Vision-Language Tracking](https://mlsanity.com/paper/2507.19875/) (2025)
- [ReasoningTrack: Chain-of-Thought Reasoning for Long-term Vision-Language Tracking](https://mlsanity.com/paper/2508.05221/) (2025)
- [Tracking by Joint Local and Global Search: A Target-aware Attention based Approach](https://mlsanity.com/paper/2106.04840/) (2021)
- [Visual Object Tracking across Diverse Data Modalities: A Review](https://mlsanity.com/paper/2412.09991/) (2024)
- [COST: Contrastive One-Stage Transformer for Vision-Language Small Object Tracking](https://mlsanity.com/paper/2504.01321/) (2025)
- [4DVLT: Dynamic Scene Understanding with Worldline-Centered Vision-Language Tracking](https://mlsanity.com/paper/2606.22631/) (2026)
- [Imperceptible Adversarial Examples for Fake Image Detection](https://mlsanity.com/paper/2106.01615/) (2021)
- [Open-World Object Detection via Discriminative Class Prototype Learning](https://mlsanity.com/paper/2302.11757/) (2023)
- [Dynamic Parsing and Updating Natural Language Specification using VLMs for Robust Vision-Language Tracking](https://mlsanity.com/paper/2606.29357/) (2026)

## Abstract (arXiv)

The tracking-by-detection framework requires a set of positive and negative training samples to learn robust tracking models for precise localization of target objects. However, existing tracking models mostly treat different samples independently while ignores the relationship information among them. In this paper, we propose a novel structure-aware deep neural network to overcome such limitations. In particular, we construct a graph to represent the pairwise relationships among training samples, and additionally take the natural language as the supervised information to learn both feature representations and classifiers robustly. To refine the states of the target and re-track the target when it is back to view from heavy occlusion and out of view, we elaborately design a novel subnetwork to learn the target-driven visual attentions from the guidance of both visual and natural language cues. Extensive experiments on five tracking benchmark datasets validated the effectiveness of our proposed method.