# AdaScan: Adaptive Scan Pooling in Deep Convolutional Neural Networks for Human Action Recognition in Videos

arXiv: https://arxiv.org/abs/1611.08240 · HTML: https://mlsanity.com/paper/1611.08240/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Nov 2016.

## Built on by (most cited first, 8 of 8)

- [AutoLabel: CLIP-based framework for Open-set Video Domain Adaptation](https://mlsanity.com/paper/2304.01110/) (2023)
- [Deep Unsupervised Key Frame Extraction for Efficient Video Classification](https://mlsanity.com/paper/2211.06742/) (2022)
- [ReSpike: Residual Frames-based Hybrid Spiking Neural Networks for Efficient Action Recognition](https://mlsanity.com/paper/2409.01564/) (2024)
- [Efficient Visual Recognition with Deep Neural Networks: A Survey on Recent Advances and New Directions](https://mlsanity.com/paper/2108.13055/) (2021)
- [Disentangled Concepts Speak Louder Than Words: Explainable Video Action Recognition](https://mlsanity.com/paper/2511.03725/) (2025)
- [Unlimited Knowledge Distillation for Action Recognition in the Dark](https://mlsanity.com/paper/2308.09327/) (2023)
- [Real-Time Activity Recognition and Intention Recognition Using a Vision-based Embedded System](https://mlsanity.com/paper/2107.12744/) (2021)
- [Making Every Frame Matter: Continuous Activity Recognition in Streaming Video via Adaptive Video Context Modeling](https://mlsanity.com/paper/2410.14993/) (2024)

## Abstract (arXiv)

We propose a novel method for temporally pooling frames in a video for the task of human action recognition. The method is motivated by the observation that there are only a small number of frames which, together, contain sufficient information to discriminate an action class present in a video, from the rest. The proposed method learns to pool such discriminative and informative frames, while discarding a majority of the non-informative frames in a single temporal scan of the video. Our algorithm does so by continuously predicting the discriminative importance of each video frame and subsequently pooling them in a deep learning framework. We show the effectiveness of our proposed pooling method on standard benchmarks where it consistently improves on baseline pooling methods, with both RGB and optical flow based Convolutional networks. Further, in combination with complementary video representations, we show results that are competitive with respect to the state-of-the-art results on two challenging and publicly available benchmark datasets.