# STF: Spatio-Temporal Fusion Module for Improving Video Object Detection

arXiv: https://arxiv.org/abs/2402.10752 · HTML: https://mlsanity.com/paper/2402.10752/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 23 of its 32 references that are in the map.
- First posted on arXiv 16 Feb 2024.

## Builds on (most cited first, 23 of 23)

- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Feature Pyramid Networks for Object Detection](https://mlsanity.com/paper/1612.03144/) (2016)
- [FCOS: Fully Convolutional One-Stage Object Detection](https://mlsanity.com/paper/1904.01355/) (2019)
- [Deep High-Resolution Representation Learning for Visual Recognition](https://mlsanity.com/paper/1908.07919/) (2019)
- [CenterNet: Keypoint Triplets for Object Detection](https://mlsanity.com/paper/1904.08189/) (2019)
- [Tracking Objects as Points](https://mlsanity.com/paper/2004.01177/) (2020)
- [Coordinate Attention for Efficient Mobile Network Design](https://mlsanity.com/paper/2103.02907/) (2021)
- [The Unmanned Aerial Vehicle Benchmark: Object Detection and Tracking](https://mlsanity.com/paper/1804.00518/) (2018)
- [Flow-Guided Feature Aggregation for Video Object Detection](https://mlsanity.com/paper/1703.10025/) (2017)
- [TCTrack: Temporal Contexts for Aerial Tracking](https://mlsanity.com/paper/2203.01885/) (2022)
- [T-CNN: Tubelets with Convolutional Neural Networks for Object Detection from Videos](https://mlsanity.com/paper/1604.02532/) (2016)
- [An Empirical Study of Spatial Attention Mechanisms in Deep Networks](https://mlsanity.com/paper/1904.05873/) (2019)
- [Object Detection in Video with Spatiotemporal Sampling Networks](https://mlsanity.com/paper/1803.05549/) (2018)
- [Video Object Detection with an Aligned Spatial-Temporal Memory](https://mlsanity.com/paper/1712.06317/) (2017)
- [Mobile Video Object Detection with Temporally-Aware Feature Maps](https://mlsanity.com/paper/1711.06368/) (2017)
- [TAda! Temporally-Adaptive Convolutions for Video Understanding](https://mlsanity.com/paper/2110.06178/) (2021)
- [Deep Learning for UAV-based Object Detection and Tracking: A Survey](https://mlsanity.com/paper/2110.12638/) (2021)
- [Revisiting Spatial Invariance with Low-Rank Local Connectivity](https://mlsanity.com/paper/2002.02959/) (2020)
- [SpotNet: Self-Attention Multi-Task Network for Object Detection](https://mlsanity.com/paper/2002.05540/) (2020)
- [Look More but Care Less in Video Recognition](https://mlsanity.com/paper/2211.09992/) (2022)
- [RN-VID: A Feature Fusion Architecture for Video Object Detection](https://mlsanity.com/paper/2003.10898/) (2020)
- [Scene Understanding for Autonomous Driving](https://mlsanity.com/paper/2105.04905/) (2021)
- [FFAVOD: Feature Fusion Architecture for Video Object Detection](https://mlsanity.com/paper/2109.07298/) (2021)

## Abstract (arXiv)

Consecutive frames in a video contain redundancy, but they may also contain relevant complementary information for the detection task. The objective of our work is to leverage this complementary information to improve detection. Therefore, we propose a spatio-temporal fusion framework (STF). We first introduce multi-frame and single-frame attention modules that allow a neural network to share feature maps between nearby frames to obtain more robust object representations. Second, we introduce a dual-frame fusion module that merges feature maps in a learnable manner to improve them. Our evaluation is conducted on three different benchmarks including video sequences of moving road users. The performed experiments demonstrate that the proposed spatio-temporal fusion module leads to improved detection performance compared to baseline object detectors. Code is available at https://github.com/noreenanwar/STF-module