# We don't Need Thousand Proposalscolon Single Shot Actor-Action Detection in Videos

arXiv: https://arxiv.org/abs/2011.10927 · HTML: https://mlsanity.com/paper/2011.10927/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 Nov 2020.

## Built on by (most cited first, 5 of 5)

- [Video Action Detection: Analysing Limitations and Challenges](https://mlsanity.com/paper/2204.07892/) (2022)
- [Exploiting Instance-based Mixed Sampling via Auxiliary Source Domain Supervision for Domain-adaptive Action Detection](https://mlsanity.com/paper/2209.15439/) (2022)
- [MED-VT++: Unifying Multimodal Learning with a Multiscale Encoder-Decoder Video Transformer](https://mlsanity.com/paper/2304.05930/) (2023)
- ["Knights": First Place Submission for VIPriors21 Action Recognition Challenge at ICCV 2021](https://mlsanity.com/paper/2110.07758/) (2021)
- [Dual-Path Temporal Map Optimization for Make-up Temporal Video Grounding](https://mlsanity.com/paper/2309.06176/) (2023)

## Abstract (arXiv)

We propose SSA2D, a simple yet effective end-to-end deep network for actor-action detection in videos. The existing methods take a top-down approach based on region-proposals (RPN), where the action is estimated based on the detected proposals followed by post-processing such as non-maximal suppression. While effective in terms of performance, these methods pose limitations in scalability for dense video scenes with a high memory requirement for thousands of proposals. We propose to solve this problem from a different perspective where we don't need any proposals. SSA2D is a unified network, which performs pixel level joint actor-action detection in a single-shot, where every pixel of the detected actor is assigned an action label. SSA2D has two main advantages: 1) It is a fully convolutional network which does not require any proposals and post-processing making it memory as well as time efficient, 2) It is easily scalable to dense video scenes as its memory requirement is independent of the number of actors present in the scene. We evaluate the proposed method on the Actor-Action dataset (A2D) and Video Object Relation (VidOR) dataset, demonstrating its effectiveness in multiple actors and action detection in a video. SSA2D is 11x faster during inference with comparable (sometimes better) performance and fewer network parameters when compared with the prior works.