# End-to-end Flow Correlation Tracking with Spatial-temporal Attention

arXiv: https://arxiv.org/abs/1711.01124 · HTML: https://mlsanity.com/paper/1711.01124/

## Key facts

- Cited by 21 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 3 Nov 2017.

## Built on by (most cited first, 21 of 21)

- [Transformer Tracking](https://mlsanity.com/paper/2103.15436/) (2021)
- [Transformer Meets Tracker: Exploiting Temporal Context for Robust Visual Tracking](https://mlsanity.com/paper/2103.11681/) (2021)
- [Residual Attention: A Simple but Effective Method for Multi-Label Recognition](https://mlsanity.com/paper/2108.02456/) (2021)
- [Visual Object Tracking with Discriminative Filters and Siamese Networks: A Survey and Outlook](https://mlsanity.com/paper/2112.02838/) (2021)
- [Saliency-Associated Object Tracking](https://mlsanity.com/paper/2108.03637/) (2021)
- [AVE-CLIP: AudioCLIP-based Multi-window Temporal Transformer for Audio Visual Event Localization](https://mlsanity.com/paper/2210.05060/) (2022)
- [TrTr: Visual Tracking with Transformer](https://mlsanity.com/paper/2105.03817/) (2021)
- [High-Performance Transformer Tracking](https://mlsanity.com/paper/2203.13533/) (2022)
- [Recent Advances in Embedding Methods for Multi-Object Tracking: A Survey](https://mlsanity.com/paper/2205.10766/) (2022)
- [Privacy-sensitive Objects Pixelation for Live Video Streaming](https://mlsanity.com/paper/2101.00604/) (2021)
- [Deep Gaussian Denoiser Epistemic Uncertainty and Decoupled Dual-Attention Fusion](https://mlsanity.com/paper/2101.04631/) (2021)
- [Multi-step Temporal Modeling for UAV Tracking](https://mlsanity.com/paper/2403.04363/) (2024)
- [Learning Temporal 3D Semantic Scene Completion via Optical Flow Guidance](https://mlsanity.com/paper/2502.14520/) (2025)
- [Benchmarking Deep Trackers on Aerial Videos](https://mlsanity.com/paper/2103.12924/) (2021)
- [Applications of physics-informed scientific machine learning in subsurface science: A survey](https://mlsanity.com/paper/2104.04764/) (2021)
- [Cannot See the Forest for the Trees: Aggregating Multiple Viewpoints to Better Classify Objects in Videos](https://mlsanity.com/paper/2206.02116/) (2022)
- [SpikeCodec: An End-to-end Learned Compression Framework for Spiking Camera](https://mlsanity.com/paper/2306.14108/) (2023)
- [Deep Convolutional Correlation Iterative Particle Filter for Visual Tracking](https://mlsanity.com/paper/2107.02984/) (2021)
- [Deep Reinforcement Learning for Optimal Power Flow with Renewables Using Graph Information](https://mlsanity.com/paper/2112.11461/) (2021)
- [TAPL: Dynamic Part-based Visual Tracking via Attention-guided Part Localization](https://mlsanity.com/paper/2110.13027/) (2021)
- [Enhancing short-term traffic prediction by integrating trends and fluctuations with attention mechanism](https://mlsanity.com/paper/2504.19967/) (2025)

## Abstract (arXiv)

Discriminative correlation filters (DCF) with deep convolutional features have achieved favorable performance in recent tracking benchmarks. However, most of existing DCF trackers only consider appearance features of current frame, and hardly benefit from motion and inter-frame information. The lack of temporal information degrades the tracking performance during challenges such as partial occlusion and deformation. In this work, we focus on making use of the rich flow information in consecutive frames to improve the feature representation and the tracking accuracy. Firstly, individual components, including optical flow estimation, feature extraction, aggregation and correlation filter tracking are formulated as special layers in network. To the best of our knowledge, this is the first work to jointly train flow and tracking task in a deep learning framework. Then the historical feature maps at predefined intervals are warped and aggregated with current ones by the guiding of flow. For adaptive aggregation, we propose a novel spatial-temporal attention mechanism. Extensive experiments are performed on four challenging tracking datasets: OTB2013, OTB2015, VOT2015 and VOT2016, and the proposed method achieves superior results on these benchmarks.