# AdaFocusV3: On Unified Spatial-temporal Dynamic Video Recognition

arXiv: https://arxiv.org/abs/2209.13465 · HTML: https://mlsanity.com/paper/2209.13465/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Builds on 47 of its 64 references that are in the map.
- First posted on arXiv 27 Sep 2022.

## Builds on (most cited first, 30 of 47)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [MobileNetV2: Inverted Residuals and Linear Bottlenecks](https://mlsanity.com/paper/1801.04381/) (2018)
- [Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation](https://mlsanity.com/paper/1406.1078/) (2014)
- [Quo Vadis, Action Recognition? A New Model and the Kinetics Dataset](https://mlsanity.com/paper/1705.07750/) (2017)
- [The Kinetics Human Action Video Dataset](https://mlsanity.com/paper/1705.06950/) (2017)
- [SlowFast Networks for Video Recognition](https://mlsanity.com/paper/1812.03982/) (2018)
- [ViViT: A Video Vision Transformer](https://mlsanity.com/paper/2103.15691/) (2021)
- [A Closer Look at Spatiotemporal Convolutions for Action Recognition](https://mlsanity.com/paper/1711.11248/) (2017)
- [The "something something" video database for learning and evaluating visual common sense](https://mlsanity.com/paper/1706.04261/) (2017)
- [Temporal Segment Networks: Towards Good Practices for Deep Action Recognition](https://mlsanity.com/paper/1608.00859/) (2016)
- [TSM: Temporal Shift Module for Efficient Video Understanding](https://mlsanity.com/paper/1811.08383/) (2018)
- [X3D: Expanding Architectures for Efficient Video Recognition](https://mlsanity.com/paper/2004.04730/) (2020)
- [Can Spatiotemporal 3D CNNs Retrace the History of 2D CNNs and ImageNet?](https://mlsanity.com/paper/1711.09577/) (2017)
- [Long-term Recurrent Convolutional Networks for Visual Recognition and Description](https://mlsanity.com/paper/1411.4389/) (2014)
- [Dynamic Neural Networks: A Survey](https://mlsanity.com/paper/2102.04906/) (2021)
- [Convolutional Two-Stream Network Fusion for Video Action Recognition](https://mlsanity.com/paper/1604.06573/) (2016)
- [Temporal Relational Reasoning in Videos](https://mlsanity.com/paper/1711.08496/) (2017)
- [Video Classification with Channel-Separated Convolutional Networks](https://mlsanity.com/paper/1904.02811/) (2019)
- [Beyond Short Snippets: Deep Networks for Video Classification](https://mlsanity.com/paper/1503.08909/) (2015)
- [Listen to Look: Action Recognition by Previewing Audio](https://mlsanity.com/paper/1912.04487/) (2019)
- [Not All Images are Worth 16x16 Words: Dynamic Transformers for Efficient Image Recognition](https://mlsanity.com/paper/2105.15075/) (2021)
- [STM: SpatioTemporal and Motion Encoding for Action Recognition](https://mlsanity.com/paper/1908.02486/) (2019)
- [TAM: Temporal Adaptive Module for Video Recognition](https://mlsanity.com/paper/2005.06803/) (2020)
- [SCSampler: Sampling Salient Clips from Video for Efficient Action Recognition](https://mlsanity.com/paper/1904.04289/) (2019)
- [Spatially Adaptive Computation Time for Residual Networks](https://mlsanity.com/paper/1612.02297/) (2016)
- [ECO: Efficient Convolutional Network for Online Video Understanding](https://mlsanity.com/paper/1804.09066/) (2018)
- [Implicit Semantic Data Augmentation for Deep Networks](https://mlsanity.com/paper/1909.12220/) (2019)
- [End-to-end Learning of Action Detection from Frame Glimpses in Videos](https://mlsanity.com/paper/1511.06984/) (2015)
- [Glance and Focus: a Dynamic Approach to Reducing Spatial Redundancy in Image Classification](https://mlsanity.com/paper/2010.05300/) (2020)
- [TEINet: Towards an Efficient Architecture for Video Recognition](https://mlsanity.com/paper/1911.09435/) (2019)

## Built on by (most cited first, 18 of 18)

- [GSVA: Generalized Segmentation via Multimodal Large Language Models](https://mlsanity.com/paper/2312.10103/) (2023)
- [Flash-VStream: Efficient Real-Time Understanding for Long Video Streams](https://mlsanity.com/paper/2506.23825/) (2025)
- [EfficientTrain: Exploring Generalized Curriculum Learning for Training Visual Backbones](https://mlsanity.com/paper/2211.09703/) (2022)
- [Dynamic Perceiver for Efficient Visual Recognition](https://mlsanity.com/paper/2306.11248/) (2023)
- [Efficient Diffusion Transformer with Step-wise Dynamic Attention Mediators](https://mlsanity.com/paper/2408.05710/) (2024)
- [How can objects help action recognition?](https://mlsanity.com/paper/2306.11726/) (2023)
- [Computation-efficient Deep Learning for Computer Vision: A Survey](https://mlsanity.com/paper/2308.13998/) (2023)
- [AdaNAT: Exploring Adaptive Policy for Token-Based Image Generation](https://mlsanity.com/paper/2409.00342/) (2024)
- [LAVIB: A Large-scale Video Interpolation Benchmark](https://mlsanity.com/paper/2406.09754/) (2024)
- [Uni-AdaFocus: Spatial-temporal Dynamic Computation for Video Recognition](https://mlsanity.com/paper/2412.11228/) (2024)
- [Boosted Dynamic Neural Networks](https://mlsanity.com/paper/2211.16726/) (2022)
- [A Survey on Dynamic Neural Networks: from Computer Vision to Multi-modal Sensor Fusion](https://mlsanity.com/paper/2501.07451/) (2025)
- [View while Moving: Efficient Video Recognition in Long-untrimmed Videos](https://mlsanity.com/paper/2308.04834/) (2023)
- [Audio-Visual Glance Network for Efficient Video Recognition](https://mlsanity.com/paper/2308.09322/) (2023)
- [AdaSpot: Spend Resolution Where It Matters for Precise Event Spotting](https://mlsanity.com/paper/2602.22073/) (2026)
- [MistExit: Learning to Exit for Early Mistake Detection in Procedural Videos](https://mlsanity.com/paper/2603.14252/) (2026)
- [Masked Feature Modelling: Feature Masking for the Unsupervised Pre-training of a Graph Attention Network Block for Bottom-up Video Event Recognition](https://mlsanity.com/paper/2308.12673/) (2023)
- [Clink! Chop! Thud! -- Learning Object Sounds from Real-World Interactions](https://mlsanity.com/paper/2510.02313/) (2025)

## Abstract (arXiv)

Recent research has revealed that reducing the temporal and spatial redundancy are both effective approaches towards efficient video recognition, e.g., allocating the majority of computation to a task-relevant subset of frames or the most valuable image regions of each frame. However, in most existing works, either type of redundancy is typically modeled with another absent. This paper explores the unified formulation of spatial-temporal dynamic computation on top of the recently proposed AdaFocusV2 algorithm, contributing to an improved AdaFocusV3 framework. Our method reduces the computational cost by activating the expensive high-capacity network only on some small but informative 3D video cubes. These cubes are cropped from the space formed by frame height, width, and video duration, while their locations are adaptively determined with a light-weighted policy network on a per-sample basis. At test time, the number of the cubes corresponding to each video is dynamically configured, i.e., video cubes are processed sequentially until a sufficiently reliable prediction is produced. Notably, AdaFocusV3 can be effectively trained by approximating the non-differentiable cropping operation with the interpolation of deep features. Extensive empirical results on six benchmark datasets (i.e., ActivityNet, FCVID, Mini-Kinetics, Something-Something V1&V2 and Diving48) demonstrate that our model is considerably more efficient than competitive baselines.