mlsanityData to 7 Oct 2026

AdaFrame: Adaptive Frame Selection for Fast Video Recognition

arXiv 1811.12432 ↗First posted 29 Nov 2018cs.CVcited by 134

Not on the map. Why?

Abstract (arXiv)

We present AdaFrame, a framework that adaptively selects relevant frames on a per-input basis for fast video recognition. AdaFrame contains a Long Short-Term Memory network augmented with a global memory that provides context information for searching which frames to use over time. Trained with policy gradient methods, AdaFrame generates a prediction, determines which frame to observe next, and computes the utility, i.e., expected future rewards, of seeing more frames at each time step. At testing time, AdaFrame exploits predicted utilities to achieve adaptive lookahead inference such that the overall computational costs are reduced without incurring a decrease in accuracy. Extensive experiments are conducted on two large-scale video benchmarks, FCVID and ActivityNet. AdaFrame matches the performance of using all frames with only 8.21 and 8.65 frames on FCVID and ActivityNet, respectively. We further qualitatively demonstrate learned frame usage can indicate the difficulty of making classification decisions; easier samples need fewer frames while harder ones require more, both at instance-level within the same class and at class-level among different categories.

Key facts. Cited by 134 papers in the map (data through 7 Oct 2026). First posted on arXiv 29 Nov 2018.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 29; 2022: 25; 2023: 26; 2024: 12; 2025: 18; 2026: 24.

134 papers.

Sort:

By area:

  1. 2021Less is More: ClipBERT for Video-and-Language Learning via Sparse Samplingcited by 484
  2. 2021Dynamic Neural Networks: A Surveycited by 405
  3. 2024MA-LMM: Memory-Augmented Large Multimodal Model for Long-Term Video Understandingcited by 223
  4. 2022Revisiting the "Video" in Video-Language Understandingcited by 167
  5. 2023Selective Structured State-Spaces for Long-Form Video Understandingcited by 105
  6. 2021SUTD-TrafficQA: A Question Answering Benchmark and an Efficient Network for Video Reasoning over Traffic Eventscited by 84
  7. 2025Q-Frame: Query-aware Frame Selection and Multi-Resolution Adaptation for Video-LLMscited by 72
  8. 2021FrameExit: Conditional Early Exiting for Efficient Video Recognitioncited by 59
  9. 2021Efficient Video Transformers with Spatial-Temporal Token Selectioncited by 58
  10. 2023A Dynamic Multi-Scale Voxel Flow Network for Video Predictioncited by 53
  11. 2021MGSampler: An Explainable Sampling Strategy for Video Action Recognitioncited by 49
  12. 2022Rethinking Video ViTs: Sparse Video Tubes for Joint Image and Video Learningcited by 49
38 more
  1. 2021AdaFuse: Adaptive Temporal Fusion Network for Efficient Action Recognitioncited by 48
  2. 2021AdaMML: Adaptive Multi-Modal Learning for Efficient Video Recognitioncited by 36
  3. 2021Dynamic Network Quantization for Efficient Video Inferencecited by 36
  4. 2022Reading-strategy Inspired Visual Representation Learning for Text-to-Video Retrievalcited by 34
  5. 2025Scaling Video-Language Models to 10K Frames via Hierarchical Differential Distillationcited by 33
  6. 2022MAR: Masked Autoencoders for Efficient Action Recognitioncited by 29
  7. 2021Coarse-Fine Networks for Temporal Activity Detection in Videoscited by 28
  8. 2021Task2Sim : Towards Effective Pre-training and Transfer from Synthetic Datacited by 26
  9. 2021No frame left behind: Full Video Action Recognitioncited by 24
  10. 2022Dynamic Spatial Sparsification for Efficient Vision Transformers and Convolutional Neural Networkscited by 23
  11. 2023Prune Spatio-temporal Tokens by Semantic-aware Temporal Accumulationcited by 23
  12. 2022OCSampler: Compressing Videos to One Clip with Single-step Samplingcited by 22
  13. 2021AdaSGN: Adapting Joint Number and Model Size for Efficient Skeleton-Based Action Recognitioncited by 21
  14. 2021Can An Image Classifier Suffice For Action Recognition?cited by 20
  15. 2023Training a Large Video Model on a Single Machine in a Daycited by 20
  16. 2023Efficient Video Action Detection with Token Dropout and Context Refinementcited by 18
  17. 2024DyFADet: Dynamic Feature Aggregation for Temporal Action Detectioncited by 18
  18. 2021Temporal Action Proposal Generation with Transformerscited by 17
  19. 2022Efficient Deep Visual and Inertial Odometry with Adaptive Visual Modality Selectioncited by 17
  20. 2022Dynamic Spatio-Temporal Specialization Learning for Fine-Grained Action Recognitioncited by 17
  21. 2023SpotEM: Efficient Video Search for Episodic Memorycited by 17
  22. 2022DeepPerform: An Efficient Approach for Performance Testing of Resource-Constrained Neural Networkscited by 16
  23. 2023Watt For What: Rethinking Deep Learning's Energy-Performance Relationshipcited by 16
  24. 2021VA-RED2: Video Adaptive Redundancy Reductioncited by 15
  25. 2022Gate-Shift-Fuse for Video Action Recognitioncited by 15
  26. 2022NSNet: Non-saliency Suppression Sampler for Efficient Video Recognitioncited by 15
  27. 2022EVEREST: Efficient Masked Video Autoencoder by Removing Redundant Spatiotemporal Tokenscited by 15
  28. 2023Computation-efficient Deep Learning for Computer Vision: A Surveycited by 14
  29. 2023Grounding-Prompter: Prompting LLM with Multimodal Information for Temporal Sentence Grounding in Long Videoscited by 14
  30. 2024Enhancing Long Video Understanding via Hierarchical Event-Based Memorycited by 14
  31. 2022The Wisdom of Crowds: Temporal Progressive Attention for Early Action Predictioncited by 13
  32. 2021MutualNet: Adaptive ConvNet via Mutual Learning from Different Model Configurationscited by 12
  33. 2023What Can Simple Arithmetic Operations Do for Temporal Modeling?cited by 12
  34. 2022Learning a Condensed Frame for Memory-Efficient Video Class-Incremental Learningcited by 11
  35. 2022Active Acquisition for Multimodal Temporal Data: A Challenging Decision-Making Taskcited by 11
  36. 2024LAVIB: A Large-scale Video Interpolation Benchmarkcited by 11
  37. 2023AdaBrowse: Adaptive Video Browser for Efficient Continuous Sign Language Recognitioncited by 10
  38. 2022Task-adaptive Spatial-Temporal Video Sampler for Few-shot Action Recognitioncited by 9

50 most cited of 134 All 134 (JSON)