# Actor Conditioned Attention Maps for Video Action Detection

arXiv: https://arxiv.org/abs/1812.11631 · HTML: https://mlsanity.com/paper/1812.11631/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 Dec 2018.

## Built on by (most cited first, 15 of 15)

- [What to look at and where: Semantic and Spatial Refined Transformer for detecting human-object interactions](https://mlsanity.com/paper/2204.00746/) (2022)
- [Deep Learning-based Action Detection in Untrimmed Videos: A Survey](https://mlsanity.com/paper/2110.00111/) (2021)
- [GTNet:Guided Transformer Network for Detecting Human-Object Interactions](https://mlsanity.com/paper/2108.00596/) (2021)
- [A Survey on Deep Learning-based Spatio-temporal Action Detection](https://mlsanity.com/paper/2308.01618/) (2023)
- [SCB-Dataset3: A Benchmark for Detecting Student Classroom Behavior](https://mlsanity.com/paper/2310.02522/) (2023)
- [Multiscale Vision Transformers meet Bipartite Matching for efficient single-stage Action Localization](https://mlsanity.com/paper/2312.17686/) (2023)
- [Identity-aware Graph Memory Network for Action Detection](https://mlsanity.com/paper/2108.11559/) (2021)
- [DDS: Decoupled Dynamic Scene-Graph Generation Network](https://mlsanity.com/paper/2301.07666/) (2023)
- [CycleACR: Cycle Modeling of Actor-Context Relations for Video Action Detection](https://mlsanity.com/paper/2303.16118/) (2023)
- [BEE-NET: A deep neural network to identify in-the-wild Bodily Expression of Emotions](https://mlsanity.com/paper/2402.13955/) (2024)
- [Position: Life-Logging Video Streams Make the Privacy-Utility Trade-off Inevitable](https://mlsanity.com/paper/2605.10404/) (2026)
- [Spot What Matters: Learning Context Using Graph Convolutional Networks for Weakly-Supervised Action Detection](https://mlsanity.com/paper/2107.13648/) (2021)
- [whu-nercms at trecvid2021:instance search task](https://mlsanity.com/paper/2111.00228/) (2021)
- [Actor-identified Spatiotemporal Action Detection -- Detecting Who Is Doing What in Videos](https://mlsanity.com/paper/2208.12940/) (2022)
- [Student Classroom Behavior Detection based on Spatio-Temporal Network and Multi-Model Fusion](https://mlsanity.com/paper/2310.16267/) (2023)

## Abstract (arXiv)

While observing complex events with multiple actors, humans do not assess each actor separately, but infer from the context. The surrounding context provides essential information for understanding actions. To this end, we propose to replace region of interest(RoI) pooling with an attention module, which ranks each spatio-temporal region's relevance to a detected actor instead of cropping. We refer to these as Actor-Conditioned Attention Maps (ACAM), which amplify/dampen the features extracted from the entire scene. The resulting actor-conditioned features focus the model on regions that are relevant to the conditioned actor. For actor localization, we leverage pre-trained object detectors, which transfer better. The proposed model is efficient and our action detection pipeline achieves near real-time performance. Experimental results on AVA 2.1 and JHMDB demonstrate the effectiveness of attention maps, with improvements of 7 mAP on AVA and 4 mAP on JHMDB.