# Sequence Level Semantics Aggregation for Video Object Detection

arXiv: https://arxiv.org/abs/1907.06390 · HTML: https://mlsanity.com/paper/1907.06390/

## Key facts

- Cited by 87 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Jul 2019.

## Built on by (most cited first, 30 of 87)

- [Global Tracking Transformers](https://mlsanity.com/paper/2203.13250/) (2022)
- [Crossover Learning for Fast Online Video Instance Segmentation](https://mlsanity.com/paper/2104.05970/) (2021)
- [TransVOD: End-to-End Video Object Detection with Spatial-Temporal Transformers](https://mlsanity.com/paper/2201.05047/) (2022)
- [3D-MAN: 3D Multi-frame Attention Network for Object Detection](https://mlsanity.com/paper/2103.16054/) (2021)
- [Scene Graph Generation: A Comprehensive Survey](https://mlsanity.com/paper/2201.00443/) (2022)
- [MPPNet: Multi-Frame Feature Intertwining with Proxy Points for 3D Temporal Object Detection](https://mlsanity.com/paper/2205.05979/) (2022)
- [Temporal RoI Align for Video Object Recognition](https://mlsanity.com/paper/2109.03495/) (2021)
- [End-to-End Video Object Detection with Spatial-Temporal Transformers](https://mlsanity.com/paper/2105.10920/) (2021)
- [TF-Blender: Temporal Feature Blender for Video Object Detection](https://mlsanity.com/paper/2108.05821/) (2021)
- [A New Dataset and A Baseline Model for Breast Lesion Detection in Ultrasound Videos](https://mlsanity.com/paper/2207.00141/) (2022)
- [TransFlow: Transformer as Flow Learner](https://mlsanity.com/paper/2304.11523/) (2023)
- [Regular Polytope Networks](https://mlsanity.com/paper/2103.15632/) (2021)
- [FeatEnHancer: Enhancing Hierarchical Features for Object Detection and Beyond Under Low-Light Vision](https://mlsanity.com/paper/2308.03594/) (2023)
- [Dogfight: Detecting Drones from Drones Videos](https://mlsanity.com/paper/2103.17242/) (2021)
- [YOLOV: Making Still Image Object Detectors Great at Video Object Detection](https://mlsanity.com/paper/2208.09686/) (2022)
- [PTSEFormer: Progressive Temporal-Spatial Enhanced TransFormer Towards Video Object Detection](https://mlsanity.com/paper/2209.02242/) (2022)
- [MGTANet: Encoding Sequential LiDAR Points Using Long Short-Term Motion-Guided Temporal Attention for 3D Object Detection](https://mlsanity.com/paper/2212.00442/) (2022)
- [Transformers in Small Object Detection: A Benchmark and Survey of State-of-the-Art](https://mlsanity.com/paper/2309.04902/) (2023)
- [STC: Spatio-Temporal Contrastive Learning for Video Instance Segmentation](https://mlsanity.com/paper/2202.03747/) (2022)
- [VNVC: A Versatile Neural Video Coding Framework for Efficient Human-Machine Vision](https://mlsanity.com/paper/2306.10681/) (2023)
- [MAMBA: Multi-level Aggregation via Memory Bank for Video Object Detection](https://mlsanity.com/paper/2401.09923/) (2024)
- [QueryProp: Object Query Propagation for High-Performance Video Object Detection](https://mlsanity.com/paper/2207.10959/) (2022)
- [LongShortNet: Exploring Temporal and Semantic Features Fusion in Streaming Perception](https://mlsanity.com/paper/2210.15518/) (2022)
- [FAQ: Feature Aggregated Queries for Transformer-based Video Object Detectors](https://mlsanity.com/paper/2303.08319/) (2023)
- [TransVisDrone: Spatio-Temporal Transformer for Vision-based Drone-to-Drone Detection in Aerial Videos](https://mlsanity.com/paper/2210.08423/) (2022)
- [CML-MOTS: Collaborative Multi-task Learning for Multi-Object Tracking and Segmentation](https://mlsanity.com/paper/2311.00987/) (2023)
- [Monocular 3D lane detection for Autonomous Driving: Recent Achievements, Challenges, and Outlooks](https://mlsanity.com/paper/2404.06860/) (2024)
- [Spatio-Temporal Learnable Proposals for End-to-End Video Object Detection](https://mlsanity.com/paper/2210.02368/) (2022)
- [Global-Local MAV Detection under Challenging Conditions based on Appearance and Motion](https://mlsanity.com/paper/2312.11008/) (2023)
- [Few-Shot Video Object Detection](https://mlsanity.com/paper/2104.14805/) (2021)

## Abstract (arXiv)

Video objection detection (VID) has been a rising research direction in recent years. A central issue of VID is the appearance degradation of video frames caused by fast motion. This problem is essentially ill-posed for a single frame. Therefore, aggregating features from other frames becomes a natural choice. Existing methods rely heavily on optical flow or recurrent neural networks for feature aggregation. However, these methods emphasize more on the temporally nearby frames. In this work, we argue that aggregating features in the full-sequence level will lead to more discriminative and robust features for video object detection. To achieve this goal, we devise a novel Sequence Level Semantics Aggregation (SELSA) module. We further demonstrate the close relationship between the proposed method and the classic spectral clustering method, providing a novel view for understanding the VID problem. We test the proposed method on the ImageNet VID and the EPIC KITCHENS dataset and achieve new state-of-the-art results. Our method does not need complicated postprocessing methods such as Seq-NMS or Tubelet rescoring, which keeps the pipeline simple and clean.