# Hierarchical Attentive Recurrent Tracking

arXiv: https://arxiv.org/abs/1706.09262 · HTML: https://mlsanity.com/paper/1706.09262/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Jun 2017.

## Built on by (most cited first, 7 of 7)

- [Visual Attention Methods in Deep Learning: An In-Depth Survey](https://mlsanity.com/paper/2204.07756/) (2022)
- [Tracking Without Re-recognition in Humans and Machines](https://mlsanity.com/paper/2105.13351/) (2021)
- [Spatial Attention Improves Iterative 6D Object Pose Estimation](https://mlsanity.com/paper/2101.01659/) (2021)
- [Exploring Transferable and Robust Adversarial Perturbation Generation from the Perspective of Network Hierarchy](https://mlsanity.com/paper/2108.07033/) (2021)
- [Recurrent Attention Models with Object-centric Capsule Representation for Multi-object Recognition](https://mlsanity.com/paper/2110.04954/) (2021)
- [SaccadeCam: Adaptive Visual Attention for Monocular Depth Sensing](https://mlsanity.com/paper/2103.12981/) (2021)
- [Pay Attention to What and Where? Interpretable Feature Extractor in Vision-based Deep Reinforcement Learning](https://mlsanity.com/paper/2504.10071/) (2025)

## Abstract (arXiv)

Class-agnostic object tracking is particularly difficult in cluttered environments as target specific discriminative models cannot be learned a priori. Inspired by how the human visual cortex employs spatial attention and separate "where" and "what" processing pathways to actively suppress irrelevant visual features, this work develops a hierarchical attentive recurrent model for single object tracking in videos. The first layer of attention discards the majority of background by selecting a region containing the object of interest, while the subsequent layers tune in on visual features particular to the tracked object. This framework is fully differentiable and can be trained in a purely data driven fashion by gradient methods. To improve training convergence, we augment the loss function with terms for a number of auxiliary tasks relevant for tracking. Evaluation of the proposed model is performed on two datasets: pedestrian tracking on the KTH activity recognition dataset and the more difficult KITTI object tracking dataset.