# Object Detection in Video with Spatiotemporal Sampling Networks

arXiv: https://arxiv.org/abs/1803.05549 · HTML: https://mlsanity.com/paper/1803.05549/

## Key facts

- Cited by 57 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Mar 2018.

## Built on by (most cited first, 30 of 57)

- [Track to Detect and Segment: An Online Multi-Object Tracker](https://mlsanity.com/paper/2103.08808/) (2021)
- [Occluded Video Instance Segmentation: A Benchmark](https://mlsanity.com/paper/2102.01558/) (2021)
- [ALIKED: A Lighter Keypoint and Descriptor Extraction Network via Deformable Transformation](https://mlsanity.com/paper/2304.03608/) (2023)
- [Crossover Learning for Fast Online Video Instance Segmentation](https://mlsanity.com/paper/2104.05970/) (2021)
- [TransVOD: End-to-End Video Object Detection with Spatial-Temporal Transformers](https://mlsanity.com/paper/2201.05047/) (2022)
- [Video Instance Segmentation with a Propose-Reduce Paradigm](https://mlsanity.com/paper/2103.13746/) (2021)
- [DG-Font: Deformable Generative Networks for Unsupervised Font Generation](https://mlsanity.com/paper/2104.03064/) (2021)
- [MPPNet: Multi-Frame Feature Intertwining with Proxy Points for 3D Temporal Object Detection](https://mlsanity.com/paper/2205.05979/) (2022)
- [Attentions Help CNNs See Better: Attention-based Hybrid Image Quality Assessment Network](https://mlsanity.com/paper/2204.10485/) (2022)
- [Temporal RoI Align for Video Object Recognition](https://mlsanity.com/paper/2109.03495/) (2021)
- [Spatial Feature Calibration and Temporal Fusion for Effective One-stage Video Instance Segmentation](https://mlsanity.com/paper/2104.05606/) (2021)
- [End-to-End Video Object Detection with Spatial-Temporal Transformers](https://mlsanity.com/paper/2105.10920/) (2021)
- [Monocular 3D Object Detection with Depth from Motion](https://mlsanity.com/paper/2207.12988/) (2022)
- [TF-Blender: Temporal Feature Blender for Video Object Detection](https://mlsanity.com/paper/2108.05821/) (2021)
- [TubeFormer-DeepLab: Video Mask Transformer](https://mlsanity.com/paper/2205.15361/) (2022)
- [From Multi-View to Hollow-3D: Hallucinated Hollow-3D R-CNN for 3D Object Detection](https://mlsanity.com/paper/2107.14391/) (2021)
- [D^2Conv3D: Dynamic Dilated Convolutions for Object Segmentation in Videos](https://mlsanity.com/paper/2111.07774/) (2021)
- [FDAN: Flow-guided Deformable Alignment Network for Video Super-Resolution](https://mlsanity.com/paper/2105.05640/) (2021)
- [Detecting and Tracking Small and Dense Moving Objects in Satellite Videos: A Benchmark](https://mlsanity.com/paper/2111.12960/) (2021)
- [Real-Time Idling Vehicles Detection using Combined Audio-Visual Deep Learning](https://mlsanity.com/paper/2305.14579/) (2023)
- [Multi-frame Collaboration for Effective Endoscopic Video Polyp Detection via Spatial-Temporal Feature Transformation](https://mlsanity.com/paper/2107.03609/) (2021)
- [Color Shift Estimation-and-Correction for Image Enhancement](https://mlsanity.com/paper/2405.17725/) (2024)
- [Improving Video Instance Segmentation via Temporal Pyramid Routing](https://mlsanity.com/paper/2107.13155/) (2021)
- [STC: Spatio-Temporal Contrastive Learning for Video Instance Segmentation](https://mlsanity.com/paper/2202.03747/) (2022)
- [MAMBA: Multi-level Aggregation via Memory Bank for Video Object Detection](https://mlsanity.com/paper/2401.09923/) (2024)
- [QueryProp: Object Query Propagation for High-Performance Video Object Detection](https://mlsanity.com/paper/2207.10959/) (2022)
- [LongShortNet: Exploring Temporal and Semantic Features Fusion in Streaming Perception](https://mlsanity.com/paper/2210.15518/) (2022)
- [Boggart: Towards General-Purpose Acceleration of Retrospective Video Analytics](https://mlsanity.com/paper/2106.15315/) (2021)
- [A recurrent CNN for online object detection on raw radar frames](https://mlsanity.com/paper/2212.11172/) (2022)
- [Recent Advances in Embedding Methods for Multi-Object Tracking: A Survey](https://mlsanity.com/paper/2205.10766/) (2022)

## Abstract (arXiv)

We propose a Spatiotemporal Sampling Network (STSN) that uses deformable convolutions across time for object detection in videos. Our STSN performs object detection in a video frame by learning to spatially sample features from the adjacent frames. This naturally renders the approach robust to occlusion or motion blur in individual frames. Our framework does not require additional supervision, as it optimizes sampling locations directly with respect to object detection performance. Our STSN outperforms the state-of-the-art on the ImageNet VID dataset and compared to prior video object detection methods it uses a simpler design, and does not require optical flow data for training.