# RADNet: A Deep Neural Network Model for Robust Perception in Moving Autonomous Systems

arXiv: https://arxiv.org/abs/2205.00364 · HTML: https://mlsanity.com/paper/2205.00364/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 26 of its 39 references that are in the map.
- First posted on arXiv 30 Apr 2022.

## Builds on (most cited first, 26 of 26)

- [Feature Pyramid Networks for Object Detection](https://mlsanity.com/paper/1612.03144/) (2016)
- [Non-local Neural Networks](https://mlsanity.com/paper/1711.07971/) (2017)
- [Deformable Convolutional Networks](https://mlsanity.com/paper/1703.06211/) (2017)
- [Two-Stream Convolutional Networks for Action Recognition in Videos](https://mlsanity.com/paper/1406.2199/) (2014)
- [AVA: A Video Dataset of Spatio-temporally Localized Atomic Visual Actions](https://mlsanity.com/paper/1705.08421/) (2017)
- [Flow-Guided Feature Aggregation for Video Object Detection](https://mlsanity.com/paper/1703.10025/) (2017)
- [Detect to Track and Track to Detect](https://mlsanity.com/paper/1710.03958/) (2017)
- [Action Tubelet Detector for Spatio-Temporal Action Localization](https://mlsanity.com/paper/1705.01861/) (2017)
- [P-CNN: Pose-based CNN Features for Action Recognition](https://mlsanity.com/paper/1506.03607/) (2015)
- [Online Real-time Multiple Spatiotemporal Action Localisation and Prediction](https://mlsanity.com/paper/1611.08563/) (2016)
- [Object Detection in Video with Spatiotemporal Sampling Networks](https://mlsanity.com/paper/1803.05549/) (2018)
- [Learning Motion Patterns in Videos](https://mlsanity.com/paper/1612.07217/) (2016)
- [STEP: Spatio-Temporal Progressive Learning for Video Action Detection](https://mlsanity.com/paper/1904.09288/) (2019)
- [Finding Action Tubes](https://mlsanity.com/paper/1411.6031/) (2014)
- [Learning to track for spatio-temporal action localization](https://mlsanity.com/paper/1506.01929/) (2015)
- [A Structured Model For Action Detection](https://mlsanity.com/paper/1812.03544/) (2018)
- [TACNet: Transition-Aware Context Network for Spatio-Temporal Action Detection](https://mlsanity.com/paper/1905.13417/) (2019)
- [Deep Learning for Detecting Multiple Space-Time Action Tubes in Videos](https://mlsanity.com/paper/1608.01529/) (2016)
- [Dance with Flow: Two-in-One Stream Action Detection](https://mlsanity.com/paper/1904.00696/) (2019)
- [CoupleNet: Coupling Global Structure with Local Parts for Object Detection](https://mlsanity.com/paper/1708.02863/) (2017)
- [Action Recognition: From Static Datasets to Moving Robots](https://mlsanity.com/paper/1701.04925/) (2017)
- [AMTnet: Action-Micro-Tube Regression by End-to-end Trainable Deep Architecture](https://mlsanity.com/paper/1704.04952/) (2017)
- [An End-to-end 3D Convolutional Neural Network for Action Detection and Segmentation in Videos](https://mlsanity.com/paper/1712.01111/) (2017)
- [TraMNet - Transition Matrix Network for Efficient Action Tube Proposals](https://mlsanity.com/paper/1808.00297/) (2018)
- [Predicting Action Tubes](https://mlsanity.com/paper/1808.07712/) (2018)
- [Generic Tubelet Proposals for Action Localization](https://mlsanity.com/paper/1705.10861/) (2017)

## Abstract (arXiv)

Interactive autonomous applications require robustness of the perception engine to artifacts in unconstrained videos. In this paper, we examine the effect of camera motion on the task of action detection. We develop a novel ranking method to rank videos based on the degree of global camera motion. For the high ranking camera videos we show that the accuracy of action detection is decreased. We propose an action detection pipeline that is robust to the camera motion effect and verify it empirically. Specifically, we do actor feature alignment across frames and couple global scene features with local actor-specific features. We do feature alignment using a novel formulation of the Spatio-temporal Sampling Network (STSN) but with multi-scale offset prediction and refinement using a pyramid structure. We also propose a novel input dependent weighted averaging strategy for fusing local and global features. We show the applicability of our network on our dataset of moving camera videos with high camera motion (MOVE dataset) with a 4.1% increase in frame mAP and 17% increase in video mAP.