# Temporal RoI Align for Video Object Recognition

arXiv: https://arxiv.org/abs/2109.03495 · HTML: https://mlsanity.com/paper/2109.03495/

## Key facts

- Cited by 44 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 38 references that are in the map.
- First posted on arXiv 8 Sep 2021.

## Builds on (most cited first, 30 of 33)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [Fast R-CNN](https://mlsanity.com/paper/1504.08083/) (2015)
- [Aggregated Residual Transformations for Deep Neural Networks](https://mlsanity.com/paper/1611.05431/) (2016)
- [Non-local Neural Networks](https://mlsanity.com/paper/1711.07971/) (2017)
- [Rich feature hierarchies for accurate object detection and semantic segmentation](https://mlsanity.com/paper/1311.2524/) (2013)
- [FCOS: Fully Convolutional One-Stage Object Detection](https://mlsanity.com/paper/1904.01355/) (2019)
- [Deformable Convolutional Networks](https://mlsanity.com/paper/1703.06211/) (2017)
- [Cascade R-CNN: Delving into High Quality Object Detection](https://mlsanity.com/paper/1712.00726/) (2017)
- [FlowNet: Learning Optical Flow with Convolutional Networks](https://mlsanity.com/paper/1504.06852/) (2015)
- [Scaling Egocentric Vision: The EPIC-KITCHENS Dataset](https://mlsanity.com/paper/1804.02748/) (2018)
- [Deformable ConvNets v2: More Deformable, Better Results](https://mlsanity.com/paper/1811.11168/) (2018)
- [CenterNet: Keypoint Triplets for Object Detection](https://mlsanity.com/paper/1904.08189/) (2019)
- [R-FCN: Object Detection via Region-based Fully Convolutional Networks](https://mlsanity.com/paper/1605.06409/) (2016)
- [Video Instance Segmentation](https://mlsanity.com/paper/1905.04804/) (2019)
- [Relation Networks for Object Detection](https://mlsanity.com/paper/1711.11575/) (2017)
- [Deep Feature Flow for Video Recognition](https://mlsanity.com/paper/1611.07715/) (2016)
- [Flow-Guided Feature Aggregation for Video Object Detection](https://mlsanity.com/paper/1703.10025/) (2017)
- [Detect to Track and Track to Detect](https://mlsanity.com/paper/1710.03958/) (2017)
- [Memory Enhanced Global-Local Aggregation for Video Object Detection](https://mlsanity.com/paper/2003.12063/) (2020)
- [Sequence Level Semantics Aggregation for Video Object Detection](https://mlsanity.com/paper/1907.06390/) (2019)
- [Relation Distillation Networks for Video Object Detection](https://mlsanity.com/paper/1908.09511/) (2019)
- [Seq-NMS for Video Object Detection](https://mlsanity.com/paper/1602.08465/) (2016)
- [Towards High Performance Video Object Detection](https://mlsanity.com/paper/1711.11577/) (2017)
- [Object Detection in Video with Spatiotemporal Sampling Networks](https://mlsanity.com/paper/1803.05549/) (2018)
- [Video Object Detection with an Aligned Spatial-Temporal Memory](https://mlsanity.com/paper/1712.06317/) (2017)
- [Mobile Video Object Detection with Temporally-Aware Feature Maps](https://mlsanity.com/paper/1711.06368/) (2017)

## Built on by (most cited first, 30 of 44)

- [TransVOD: End-to-End Video Object Detection with Spatial-Temporal Transformers](https://mlsanity.com/paper/2201.05047/) (2022)
- [Learning Object State Changes in Videos: An Open-World Perspective](https://mlsanity.com/paper/2312.11782/) (2023)
- [A New Dataset and A Baseline Model for Breast Lesion Detection in Ultrasound Videos](https://mlsanity.com/paper/2207.00141/) (2022)
- [FeatEnHancer: Enhancing Hierarchical Features for Object Detection and Beyond Under Low-Light Vision](https://mlsanity.com/paper/2308.03594/) (2023)
- [YOLOV: Making Still Image Object Detectors Great at Video Object Detection](https://mlsanity.com/paper/2208.09686/) (2022)
- [PTSEFormer: Progressive Temporal-Spatial Enhanced TransFormer Towards Video Object Detection](https://mlsanity.com/paper/2209.02242/) (2022)
- [Hybrid Instance-aware Temporal Fusion for Online Video Instance Segmentation](https://mlsanity.com/paper/2112.01695/) (2021)
- [FAQ: Feature Aggregated Queries for Transformer-based Video Object Detectors](https://mlsanity.com/paper/2303.08319/) (2023)
- [Objects do not disappear: Video object detection by single-frame object location anticipation](https://mlsanity.com/paper/2308.04770/) (2023)
- [BoxMask: Revisiting Bounding Box Supervision for Video Object Detection](https://mlsanity.com/paper/2210.06008/) (2022)
- [DFA: Dynamic Feature Aggregation for Efficient Video Object Detection](https://mlsanity.com/paper/2210.00588/) (2022)
- [Flying Bird Object Detection Algorithm in Surveillance Video Based on Motion Information](https://mlsanity.com/paper/2301.01917/) (2023)
- [DarkVision: A Benchmark for Low-light Image/Video Perception](https://mlsanity.com/paper/2301.06269/) (2023)
- [Learned Scalable Video Coding For Humans and Machines](https://mlsanity.com/paper/2307.08978/) (2023)
- [A Spatial-Temporal Deformable Attention based Framework for Breast Lesion Detection in Videos](https://mlsanity.com/paper/2309.04702/) (2023)
- [Practical Video Object Detection via Feature Selection and Aggregation](https://mlsanity.com/paper/2407.19650/) (2024)
- [FBD-SV-2024: Flying Bird Object Detection Dataset in Surveillance Video](https://mlsanity.com/paper/2409.00317/) (2024)
- [OSCAR: Object Status and Contextual Awareness for Recipes to Support Non-Visual Cooking](https://mlsanity.com/paper/2503.05962/) (2025)
- [Memory Maps for Video Object Detection and Tracking on UAVs](https://mlsanity.com/paper/2303.03508/) (2023)
- [Traffic Video Object Detection using Motion Prior](https://mlsanity.com/paper/2311.10092/) (2023)
- [Mixture of Scale Experts for Alignment-free RGBT Video Object Detection and A Unified Benchmark](https://mlsanity.com/paper/2410.12143/) (2024)
- [Erasure-based Interaction Network for RGBT Video Object Detection and A Unified Benchmark](https://mlsanity.com/paper/2308.01630/) (2023)
- [A Flying Bird Object Detection Method for Surveillance Video](https://mlsanity.com/paper/2401.03749/) (2024)
- [XS-VID: A Large-Scale Benchmark for Small Object Detection and Tracking in Videos](https://mlsanity.com/paper/2407.18137/) (2024)
- [SSVOD: Semi-Supervised Video Object Detection with Sparse Annotations](https://mlsanity.com/paper/2309.01391/) (2023)
- [Source-free Domain Adaptation for Video Object Detection Under Adverse Image Conditions](https://mlsanity.com/paper/2404.15252/) (2024)
- [A Training-Free Framework for Video License Plate Tracking and Recognition with Only One-Shot](https://mlsanity.com/paper/2408.05729/) (2024)
- [Exploring Object Status Recognition for Recipe Progress Tracking in Non-Visual Cooking](https://mlsanity.com/paper/2507.03330/) (2025)
- [Object Detection Difficulty: Suppressing Over-aggregation for Faster and Better Video Object Detection](https://mlsanity.com/paper/2308.11327/) (2023)
- [FlightScope: An Experimental Comparative Review of Aircraft Detection Algorithms in Satellite Imagery](https://mlsanity.com/paper/2404.02877/) (2024)

## Abstract (arXiv)

Video object detection is challenging in the presence of appearance deterioration in certain video frames. Therefore, it is a natural choice to aggregate temporal information from other frames of the same video into the current frame. However, RoI Align, as one of the most core procedures of video detectors, still remains extracting features from a single-frame feature map for proposals, making the extracted RoI features lack temporal information from videos. In this work, considering the features of the same object instance are highly similar among frames in a video, a novel Temporal RoI Align operator is proposed to extract features from other frames feature maps for current frame proposals by utilizing feature similarity. The proposed Temporal RoI Align operator can extract temporal information from the entire video for proposals. We integrate it into single-frame video detectors and other state-of-the-art video detectors, and conduct quantitative experiments to demonstrate that the proposed Temporal RoI Align operator can consistently and significantly boost the performance. Besides, the proposed Temporal RoI Align can also be applied into video instance segmentation. Codes are available at https://github.com/open-mmlab/mmtracking