# Attentional Pooling for Action Recognition

arXiv: https://arxiv.org/abs/1711.01467 · HTML: https://mlsanity.com/paper/1711.01467/

## Key facts

- Cited by 54 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 4 Nov 2017.

## Built on by (most cited first, 30 of 54)

- [ViViT: A Video Vision Transformer](https://mlsanity.com/paper/2103.15691/) (2021)
- [VATT: Transformers for Multimodal Self-Supervised Learning from Raw Video, Audio and Text](https://mlsanity.com/paper/2104.11178/) (2021)
- [Attention Mechanisms in Computer Vision: A Survey](https://mlsanity.com/paper/2111.07624/) (2021)
- [Anticipative Video Transformer](https://mlsanity.com/paper/2106.02036/) (2021)
- [Temporal Context Aggregation Network for Temporal Action Proposal Refinement](https://mlsanity.com/paper/2103.13141/) (2021)
- [Multimodality Helps Unimodality: Cross-Modal Few-Shot Learning with Multimodal Models](https://mlsanity.com/paper/2301.06267/) (2023)
- [This Looks Like That... Does it? Shortcomings of Latent Space Prototype Interpretability in Deep Networks](https://mlsanity.com/paper/2105.02968/) (2021)
- [ViTs for SITS: Vision Transformers for Satellite Image Time Series](https://mlsanity.com/paper/2301.04944/) (2023)
- [Attention, please! A survey of Neural Attention Models in Deep Learning](https://mlsanity.com/paper/2103.16775/) (2021)
- [E2(GO)MOTION: Motion Augmented Event Stream for Egocentric Action Recognition](https://mlsanity.com/paper/2112.03596/) (2021)
- [3Mformer: Multi-order Multi-mode Transformer for Skeletal Action Recognition](https://mlsanity.com/paper/2303.14474/) (2023)
- [Class Semantics-based Attention for Action Detection](https://mlsanity.com/paper/2109.02613/) (2021)
- [Learning Self-Similarity in Space and Time as Generalized Motion for Video Action Recognition](https://mlsanity.com/paper/2102.07092/) (2021)
- [TOIST: Task Oriented Instance Segmentation Transformer with Noun-Pronoun Distillation](https://mlsanity.com/paper/2210.10775/) (2022)
- [EmoNet: A Transfer Learning Framework for Multi-Corpus Speech Emotion Recognition](https://mlsanity.com/paper/2103.08310/) (2021)
- [Efficient Image Pre-Training with Siamese Cropped Masked Autoencoders](https://mlsanity.com/paper/2403.17823/) (2024)
- [Symbol-LLM: Leverage Language Models for Symbolic System in Visual Human Activity Reasoning](https://mlsanity.com/paper/2311.17365/) (2023)
- [Gate-Shift-Fuse for Video Action Recognition](https://mlsanity.com/paper/2203.08897/) (2022)
- [RelViT: Concept-guided Vision Transformer for Visual Relational Reasoning](https://mlsanity.com/paper/2204.11167/) (2022)
- [Coarse Temporal Attention Network (CTA-Net) for Driver's Activity Recognition](https://mlsanity.com/paper/2101.06636/) (2021)
- [High-order Tensor Pooling with Attention for Action Recognition](https://mlsanity.com/paper/2110.05216/) (2021)
- [Another Vertical View: A Hierarchical Network for Heterogeneous Trajectory Prediction via Spectrums](https://mlsanity.com/paper/2304.05106/) (2023)
- [CLASTER: Clustering with Reinforcement Learning for Zero-Shot Action Recognition](https://mlsanity.com/paper/2101.07042/) (2021)
- [Temporal-attentive Covariance Pooling Networks for Video Recognition](https://mlsanity.com/paper/2110.14381/) (2021)
- [Topological Pooling on Graphs](https://mlsanity.com/paper/2303.14543/) (2023)
- [GTNet:Guided Transformer Network for Detecting Human-Object Interactions](https://mlsanity.com/paper/2108.00596/) (2021)
- [The Overlooked Classifier in Human-Object Interaction Recognition](https://mlsanity.com/paper/2112.06392/) (2021)
- [Few-Shot Fine-Grained Action Recognition via Bidirectional Attention and Contrastive Meta-Learning](https://mlsanity.com/paper/2108.06647/) (2021)
- [Learning to Recognize Actions on Objects in Egocentric Video with Attention Dictionaries](https://mlsanity.com/paper/2102.08065/) (2021)
- [Higher Order Recurrent Space-Time Transformer for Video Action Prediction](https://mlsanity.com/paper/2104.08665/) (2021)

## Abstract (arXiv)

We introduce a simple yet surprisingly powerful model to incorporate attention in action recognition and human object interaction tasks. Our proposed attention module can be trained with or without extra supervision, and gives a sizable boost in accuracy while keeping the network size and computational cost nearly the same. It leads to significant improvements over state of the art base architecture on three standard action recognition benchmarks across still images and videos, and establishes new state of the art on MPII dataset with 12.5% relative improvement. We also perform an extensive analysis of our attention module both empirically and analytically. In terms of the latter, we introduce a novel derivation of bottom-up and top-down attention as low-rank approximations of bilinear pooling methods (typically used for fine-grained classification). From this perspective, our attention formulation suggests a novel characterization of action recognition as a fine-grained recognition problem.