# TF-Blender: Temporal Feature Blender for Video Object Detection

arXiv: https://arxiv.org/abs/2108.05821 · HTML: https://mlsanity.com/paper/2108.05821/

## Key facts

- Cited by 37 papers in the map (data through 14 Oct 2026).
- Builds on 31 of its 45 references that are in the map.
- First posted on arXiv 12 Aug 2021.

## Builds on (most cited first, 30 of 31)

- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Dynamic Graph CNN for Learning on Point Clouds](https://mlsanity.com/paper/1801.07829/) (2018)
- [FlowNet: Learning Optical Flow with Convolutional Networks](https://mlsanity.com/paper/1504.06852/) (2015)
- [FlowNet 2.0: Evolution of Optical Flow Estimation with Deep Networks](https://mlsanity.com/paper/1612.01925/) (2016)
- [PointConv: Deep Convolutional Networks on 3D Point Clouds](https://mlsanity.com/paper/1811.07246/) (2018)
- [YOLACT: Real-time Instance Segmentation](https://mlsanity.com/paper/1904.02689/) (2019)
- [Video Instance Segmentation](https://mlsanity.com/paper/1905.04804/) (2019)
- [Relation-Shape Convolutional Neural Network for Point Cloud Analysis](https://mlsanity.com/paper/1904.07601/) (2019)
- [Relation Networks for Object Detection](https://mlsanity.com/paper/1711.11575/) (2017)
- [MOTS: Multi-Object Tracking and Segmentation](https://mlsanity.com/paper/1902.03604/) (2019)
- [Deep Feature Flow for Video Recognition](https://mlsanity.com/paper/1611.07715/) (2016)
- [Flow-Guided Feature Aggregation for Video Object Detection](https://mlsanity.com/paper/1703.10025/) (2017)
- [Detect to Track and Track to Detect](https://mlsanity.com/paper/1710.03958/) (2017)
- [Memory Enhanced Global-Local Aggregation for Video Object Detection](https://mlsanity.com/paper/2003.12063/) (2020)
- [Classifying, Segmenting, and Tracking Object Instances in Video with Mask Propagation](https://mlsanity.com/paper/1912.04573/) (2019)
- [STEm-Seg: Spatio-temporal Embeddings for Instance Segmentation in Videos](https://mlsanity.com/paper/2003.08429/) (2020)
- [SipMask: Spatial Information Preservation for Fast Image and Video Instance Segmentation](https://mlsanity.com/paper/2007.14772/) (2020)
- [Sequence Level Semantics Aggregation for Video Object Detection](https://mlsanity.com/paper/1907.06390/) (2019)
- [SG-Net: Spatial Granularity Network for One-Stage Video Instance Segmentation](https://mlsanity.com/paper/2103.10284/) (2021)
- [YOLACT++: Better Real-time Instance Segmentation](https://mlsanity.com/paper/1912.06218/) (2019)
- [Relation Distillation Networks for Video Object Detection](https://mlsanity.com/paper/1908.09511/) (2019)
- [Seq-NMS for Video Object Detection](https://mlsanity.com/paper/1602.08465/) (2016)
- [T-CNN: Tubelets with Convolutional Neural Networks for Object Detection from Videos](https://mlsanity.com/paper/1604.02532/) (2016)
- [Towards High Performance Video Object Detection](https://mlsanity.com/paper/1711.11577/) (2017)
- [Object Detection in Video with Spatiotemporal Sampling Networks](https://mlsanity.com/paper/1803.05549/) (2018)
- [Video Object Detection with an Aligned Spatial-Temporal Memory](https://mlsanity.com/paper/1712.06317/) (2017)
- [Object Detection from Video Tubelets with Convolutional Neural Networks](https://mlsanity.com/paper/1604.04053/) (2016)
- [DenserNet: Weakly Supervised Visual Localization Using Multi-scale Feature Aggregation](https://mlsanity.com/paper/2012.02366/) (2020)
- [Visual Localization for Autonomous Driving: Mapping the Accurate Location in the City Maze](https://mlsanity.com/paper/2008.05678/) (2020)
- [Hierarchical Attention Fusion for Geo-Localization](https://mlsanity.com/paper/2102.09186/) (2021)

## Built on by (most cited first, 30 of 37)

- [TransVOD: End-to-End Video Object Detection with Spatial-Temporal Transformers](https://mlsanity.com/paper/2201.05047/) (2022)
- [Physical Attack on Monocular Depth Estimation with Optimal Adversarial Patches](https://mlsanity.com/paper/2207.04718/) (2022)
- [TransFlow: Transformer as Flow Learner](https://mlsanity.com/paper/2304.11523/) (2023)
- [Pessimism in the Face of Confounders: Provably Efficient Offline Reinforcement Learning in Partially Observable Markov Decision Processes](https://mlsanity.com/paper/2205.13589/) (2022)
- [Learning Equivariant Segmentation with Instance-Unique Querying](https://mlsanity.com/paper/2210.00911/) (2022)
- [LongShortNet: Exploring Temporal and Semantic Features Fusion in Streaming Perception](https://mlsanity.com/paper/2210.15518/) (2022)
- [Solve the Puzzle of Instance Segmentation in Videos: A Weakly Supervised Framework with Spatio-Temporal Collaboration](https://mlsanity.com/paper/2212.07592/) (2022)
- [FAQ: Feature Aggregated Queries for Transformer-based Video Object Detectors](https://mlsanity.com/paper/2303.08319/) (2023)
- [Fusion is Not Enough: Single Modal Attacks on Fusion Models for 3D Object Detection](https://mlsanity.com/paper/2304.14614/) (2023)
- [DG-Labeler and DGL-MOTS Dataset: Boost the Autonomous Driving Perception](https://mlsanity.com/paper/2110.07790/) (2021)
- [CML-MOTS: Collaborative Multi-task Learning for Multi-Object Tracking and Segmentation](https://mlsanity.com/paper/2311.00987/) (2023)
- [Spatio-Temporal Learnable Proposals for End-to-End Video Object Detection](https://mlsanity.com/paper/2210.02368/) (2022)
- [Towards Unbiased Label Distribution Learning for Facial Pose Estimation Using Anisotropic Spherical Gaussian](https://mlsanity.com/paper/2208.09122/) (2022)
- [Objects do not disappear: Video object detection by single-frame object location anticipation](https://mlsanity.com/paper/2308.04770/) (2023)
- [Dynamic Proposals for Efficient Object Detection](https://mlsanity.com/paper/2207.05252/) (2022)
- [BoxMask: Revisiting Bounding Box Supervision for Video Object Detection](https://mlsanity.com/paper/2210.06008/) (2022)
- [Learning Dynamic Query Combinations for Transformer-based Object Detection and Segmentation](https://mlsanity.com/paper/2307.12239/) (2023)
- [ClusterFormer: Clustering As A Universal Visual Learner](https://mlsanity.com/paper/2309.13196/) (2023)
- [DFA: Dynamic Feature Aggregation for Efficient Video Object Detection](https://mlsanity.com/paper/2210.00588/) (2022)
- [Efficient One-stage Video Object Detection by Exploiting Temporal Consistency](https://mlsanity.com/paper/2402.09241/) (2024)
- [A Novel Long-term Iterative Mining Scheme for Video Salient Object Detection](https://mlsanity.com/paper/2206.09564/) (2022)
- [Basketball-SORT: An Association Method for Complex Multi-object Occlusion Problems in Basketball Multi-object Tracking](https://mlsanity.com/paper/2406.19655/) (2024)
- [Adverse Weather Optical Flow: Cumulative Homogeneous-Heterogeneous Adaptation](https://mlsanity.com/paper/2409.17001/) (2024)
- [Identity-Consistent Aggregation for Video Object Detection](https://mlsanity.com/paper/2308.07737/) (2023)
- [Explore Spatio-temporal Aggregation for Insubstantial Object Detection: Benchmark Dataset and Baseline](https://mlsanity.com/paper/2206.11459/) (2022)
- [SSVOD: Semi-Supervised Video Object Detection with Sparse Annotations](https://mlsanity.com/paper/2309.01391/) (2023)
- [CodingHomo: Bootstrapping Deep Homography With Video Coding](https://mlsanity.com/paper/2504.12165/) (2025)
- [Hard Region Aware Network for Remote Sensing Change Detection](https://mlsanity.com/paper/2305.19513/) (2023)
- [Holistic Prototype Attention Network for Few-Shot VOS](https://mlsanity.com/paper/2307.07933/) (2023)
- [Object Detection Difficulty: Suppressing Over-aggregation for Faster and Better Video Object Detection](https://mlsanity.com/paper/2308.11327/) (2023)

## Abstract (arXiv)

Video objection detection is a challenging task because isolated video frames may encounter appearance deterioration, which introduces great confusion for detection. One of the popular solutions is to exploit the temporal information and enhance per-frame representation through aggregating features from neighboring frames. Despite achieving improvements in detection, existing methods focus on the selection of higher-level video frames for aggregation rather than modeling lower-level temporal relations to increase the feature representation. To address this limitation, we propose a novel solution named TF-Blender,which includes three modules: 1) Temporal relation mod-els the relations between the current frame and its neighboring frames to preserve spatial information. 2). Feature adjustment enriches the representation of every neigh-boring feature map; 3) Feature blender combines outputs from the first two modules and produces stronger features for the later detection tasks. For its simplicity, TF-Blender can be effortlessly plugged into any detection network to improve detection behavior. Extensive evaluations on ImageNet VID and YouTube-VIS benchmarks indicate the performance guarantees of using TF-Blender on recent state-of-the-art methods.