LiteEval: A Coarse-to-Fine Framework for Resource Efficient Video Recognition
Not on the map. Why?
Abstract (arXiv)
This paper presents LiteEval, a simple yet effective coarse-to-fine framework for resource efficient video recognition, suitable for both online and offline scenarios. Exploiting decent yet computationally efficient features derived at a coarse scale with a lightweight CNN model, LiteEval dynamically decides on-the-fly whether to compute more powerful features for incoming video frames at a finer scale to obtain more details. This is achieved by a coarse LSTM and a fine LSTM operating cooperatively, as well as a conditional gating module to learn when to allocate more computation. Extensive experiments are conducted on two large-scale video benchmarks, FCVID and ActivityNet, and the results demonstrate LiteEval requires substantially less computation while offering excellent classification accuracy for both online and offline predictions.
Key facts. Cited by 65 papers in the map (data through 7 Oct 2026). First posted on arXiv 3 Dec 2019.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2021: 25; 2022: 15; 2023: 11; 2024: 9; 2025: 2; 2026: 3.
65 papers.
Sort:
By area:
- 2021Dynamic Neural Networks: A Surveycited by 405
- 2024VideoTree: Adaptive Tree-based Video Representation for LLM Reasoning on Long Videoscited by 242
- 2021AdaViT: Adaptive Vision Transformers for Efficient Image Recognitioncited by 228
- 2021BEVT: BERT Pretraining of Video Transformerscited by 169
- 2022DiffPose: Toward More Reliable 3D Pose Estimationcited by 99
- 2021SUTD-TrafficQA: A Question Answering Benchmark and an Efficient Network for Video Reasoning over Traffic Eventscited by 84
- 2021Adaptive Focus for Efficient Video Recognitioncited by 76
- 2022SVFormer: Semi-supervised Video Transformer for Action Recognitioncited by 67
- 2021FrameExit: Conditional Early Exiting for Efficient Video Recognitioncited by 59
- 2021Efficient Video Transformers with Spatial-Temporal Token Selectioncited by 58
- 2021MGSampler: An Explainable Sampling Strategy for Video Action Recognitioncited by 49
- 2021AdaFuse: Adaptive Temporal Fusion Network for Efficient Action Recognitioncited by 48
38 more
- 2021AdaFocus V2: End-to-End Training of Spatial Dynamic Networks for Video Recognitioncited by 37
- 2021AdaMML: Adaptive Multi-Modal Learning for Efficient Video Recognitioncited by 36
- 2021Dynamic Network Quantization for Efficient Video Inferencecited by 36
- 2022Reading-strategy Inspired Visual Representation Learning for Text-to-Video Retrievalcited by 34
- 2021Skip-Convolutions for Efficient Video Processingcited by 32
- 2022MAR: Masked Autoencoders for Efficient Action Recognitioncited by 29
- 2021Coarse-Fine Networks for Temporal Activity Detection in Videoscited by 28
- 2021No frame left behind: Full Video Action Recognitioncited by 24
- 2023Text-Conditioned Resampler For Long Form Video Understandingcited by 24
- 2022OCSampler: Compressing Videos to One Clip with Single-step Samplingcited by 22
- 2021AdaSGN: Adapting Joint Number and Model Size for Efficient Skeleton-Based Action Recognitioncited by 21
- 2021Can An Image Classifier Suffice For Action Recognition?cited by 20
- 2022AdaFocusV3: On Unified Spatial-temporal Dynamic Video Recognitioncited by 18
- 2021EAN: Event Adaptive Network for Enhanced Action Recognitioncited by 16
- 2022Gate-Shift-Fuse for Video Action Recognitioncited by 15
- 2022NSNet: Non-saliency Suppression Sampler for Efficient Video Recognitioncited by 15
- 2023Computation-efficient Deep Learning for Computer Vision: A Surveycited by 14
- 2023Eventful Transformers: Leveraging Temporal Redundancy in Vision Transformerscited by 13
- 2024LF-ViT: Reducing Spatial Redundancy in Vision Transformer for Efficient Image Recognitioncited by 13
- 2021MutualNet: Adaptive ConvNet via Mutual Learning from Different Model Configurationscited by 12
- 2023AdaBrowse: Adaptive Video Browser for Efficient Continuous Sign Language Recognitioncited by 10
- 2021Coarse-to-Fine Curriculum Learningcited by 9
- 2022ERA: Expert Retrieval and Assembly for Early Action Predictioncited by 9
- 2022Temporal Saliency Query Network for Efficient Video Recognitioncited by 9
- 2023System-status-aware Adaptive Network for Online Streaming Video Understandingcited by 9
- 2024Uni-AdaFocus: Spatial-temporal Dynamic Computation for Video Recognitioncited by 9
- 2022Rethinking Resolution in the Context of Efficient Video Recognitioncited by 8
- 2022A Visual Active Search Framework for Geospatial Explorationcited by 8
- 2023A Partially Supervised Reinforcement Learning Framework for Visual Active Searchcited by 8
- 2023Revisiting Kernel Temporal Segmentation as an Adaptive Tokenizer for Long-form Video Understandingcited by 7
- 2022Look More but Care Less in Video Recognitioncited by 6
- 2021Event Neural Networkscited by 5
- 2025D'ej`a Vu: Efficient Video-Language Query Engine with Learning-based Inter-Frame Computation Reusecited by 5
- 2026Fewer Steps, Better Performance: Efficient Cross-Modal Clip Trimming for Video Moment Retrieval Using Languagecited by 5
- 2022ViGAT: Bottom-up event recognition and explanation in video using factorized graph attention networkcited by 4
- 2023View while Moving: Efficient Video Recognition in Long-untrimmed Videoscited by 4
- 2023Audio-Visual Glance Network for Efficient Video Recognitioncited by 4
- 2021Modeling Temporal Concept Receptive Field Dynamically for Untrimmed Video Analysiscited by 3
50 most cited of 65 All 65 (JSON)
- 2026Why Is Video Still So Expensive? A Survey of Inference-Efficiency Mechanisms in Video and Audiovisual LLMs9 Sep 2026
- 2026Fewer Steps, Better Performance: Efficient Cross-Modal Clip Trimming for Video Moment Retrieval Using Language28 May 2026
- 2026DiffVAS: Diffusion-Guided Visual Active Search in Partially Observable Environments15 May 2026
- 2025Two-Stage Framework for Efficient UAV-Based Wildfire Video Analysis with Adaptive Compression and Fire Source Detection22 Aug 2025
- 2025D'ej`a Vu: Efficient Video-Language Query Engine with Learning-based Inter-Frame Computation Reuse17 Jun 2025
- 2024Uni-AdaFocus: Spatial-temporal Dynamic Computation for Video Recognition15 Dec 2024
- 2024Scalable Frame Sampling for Video Classification: A Semi-Optimal Policy Approach with Reduced Search Space9 Sep 2024
- 2024Streamlining Forest Wildfire Surveillance: AI-Enhanced UAVs Utilizing the FLAME Aerial Video Dataset for Lightweight and Efficient Monitoring31 Aug 2024
- 2024Task-Aware Dynamic Transformer for Efficient Arbitrary-Scale Image Super-Resolution16 Aug 2024
- 2024VideoTree: Adaptive Tree-based Video Representation for LLM Reasoning on Long Videos29 May 2024
- 2024Learning Discriminative Spatio-temporal Representations for Semi-supervised Action Recognition25 Apr 2024
- 2024Successive Refinement in Large-Scale Computation: Advancing Model Inference Applications11 Feb 2024
38 more
- 2024HaltingVT: Adaptive Token Halting Transformer for Efficient Video Recognition10 Jan 2024
- 2024LF-ViT: Reducing Spatial Redundancy in Vision Transformer for Efficient Image Recognition8 Jan 2024
- 2023Text-Conditioned Resampler For Long Form Video Understanding19 Dec 2023
- 2023A Partially Supervised Reinforcement Learning Framework for Visual Active Search15 Oct 2023
- 2023Revisiting Kernel Temporal Segmentation as an Adaptive Tokenizer for Long-form Video Understanding20 Sep 2023
- 2023Differentiable Resolution Compression and Alignment for Efficient Video Classification and Retrieval15 Sep 2023
- 2023Computation-efficient Deep Learning for Computer Vision: A Survey27 Aug 2023
- 2023Eventful Transformers: Leveraging Temporal Redundancy in Vision Transformers25 Aug 2023
- 2023Audio-Visual Glance Network for Efficient Video Recognition18 Aug 2023
- 2023AdaBrowse: Adaptive Video Browser for Efficient Continuous Sign Language Recognition16 Aug 2023
- 2023View while Moving: Efficient Video Recognition in Long-untrimmed Videos9 Aug 2023
- 2023Search-Map-Search: A Frame Selection Paradigm for Action Recognition20 Apr 2023
- 2023System-status-aware Adaptive Network for Online Streaming Video Understanding28 Mar 2023
- 2022DiffPose: Toward More Reliable 3D Pose Estimation30 Nov 2022
- 2022A Visual Active Search Framework for Geospatial Exploration28 Nov 2022
- 2022SVFormer: Semi-supervised Video Transformer for Action Recognition23 Nov 2022
- 2022Look More but Care Less in Video Recognition18 Nov 2022
- 2022AdaFocusV3: On Unified Spatial-temporal Dynamic Video Recognition27 Sep 2022
- 2022Rethinking Resolution in the Context of Efficient Video Recognition26 Sep 2022
- 2022MAR: Masked Autoencoders for Efficient Action Recognition24 Jul 2022
- 2022NSNet: Non-saliency Suppression Sampler for Efficient Video Recognition21 Jul 2022
- 2022Temporal Saliency Query Network for Efficient Video Recognition21 Jul 2022
- 2022ViGAT: Bottom-up event recognition and explanation in video using factorized graph attention network20 Jul 2022
- 2022ERA: Expert Retrieval and Assembly for Early Action Prediction20 Jul 2022
- 2022Gate-Shift-Fuse for Video Action Recognition16 Mar 2022
- 2022Reading-strategy Inspired Visual Representation Learning for Text-to-Video Retrieval23 Jan 2022
- 2022CISRNet: Compressed Image Super-Resolution Network16 Jan 2022
- 2022OCSampler: Compressing Videos to One Clip with Single-step Sampling12 Jan 2022
- 2021AdaFocus V2: End-to-End Training of Spatial Dynamic Networks for Video Recognition28 Dec 2021
- 2021BEVT: BERT Pretraining of Video Transformers2 Dec 2021
- 2021Event Neural Networks2 Dec 2021
- 2021AdaViT: Adaptive Vision Transformers for Efficient Image Recognition30 Nov 2021
- 2021Modeling Temporal Concept Receptive Field Dynamically for Untrimmed Video Analysis23 Nov 2021
- 2021Efficient Video Transformers with Spatial-Temporal Token Selection23 Nov 2021
- 2021Unsupervised Action Localization Crop in Video Retargeting for 3D ConvNets14 Nov 2021
- 2021Dynamic Network Quantization for Efficient Video Inference23 Aug 2021
- 2021Rapid Elastic Architecture Search under Specialized Classes and Resource Constraints3 Aug 2021
- 2021EAN: Event Adaptive Network for Enhanced Action Recognition22 Jul 2021
50 newest of 65 All 65 (JSON)
- 2021Dynamic Neural Networks: A Surveycited by 405
- 2021AdaViT: Adaptive Vision Transformers for Efficient Image Recognitioncited by 228
- 2021BEVT: BERT Pretraining of Video Transformerscited by 169
- 2022DiffPose: Toward More Reliable 3D Pose Estimationcited by 99
- 2021SUTD-TrafficQA: A Question Answering Benchmark and an Efficient Network for Video Reasoning over Traffic Eventscited by 84
- 2021Adaptive Focus for Efficient Video Recognitioncited by 76
- 2022SVFormer: Semi-supervised Video Transformer for Action Recognitioncited by 67
- 2021FrameExit: Conditional Early Exiting for Efficient Video Recognitioncited by 59
- 2021Efficient Video Transformers with Spatial-Temporal Token Selectioncited by 58
- 2021MGSampler: An Explainable Sampling Strategy for Video Action Recognitioncited by 49
- 2021AdaFuse: Adaptive Temporal Fusion Network for Efficient Action Recognitioncited by 48
- 2021AdaFocus V2: End-to-End Training of Spatial Dynamic Networks for Video Recognitioncited by 37
12 most cited of 63 All 63 (JSON)
- 2026Why Is Video Still So Expensive? A Survey of Inference-Efficiency Mechanisms in Video and Audiovisual LLMs9 Sep 2026
- 2026Fewer Steps, Better Performance: Efficient Cross-Modal Clip Trimming for Video Moment Retrieval Using Language28 May 2026
- 2026DiffVAS: Diffusion-Guided Visual Active Search in Partially Observable Environments15 May 2026
- 2025Two-Stage Framework for Efficient UAV-Based Wildfire Video Analysis with Adaptive Compression and Fire Source Detection22 Aug 2025
- 2025D'ej`a Vu: Efficient Video-Language Query Engine with Learning-based Inter-Frame Computation Reuse17 Jun 2025
- 2024Uni-AdaFocus: Spatial-temporal Dynamic Computation for Video Recognition15 Dec 2024
- 2024Scalable Frame Sampling for Video Classification: A Semi-Optimal Policy Approach with Reduced Search Space9 Sep 2024
- 2024Streamlining Forest Wildfire Surveillance: AI-Enhanced UAVs Utilizing the FLAME Aerial Video Dataset for Lightweight and Efficient Monitoring31 Aug 2024
- 2024Task-Aware Dynamic Transformer for Efficient Arbitrary-Scale Image Super-Resolution16 Aug 2024
- 2024Successive Refinement in Large-Scale Computation: Advancing Model Inference Applications11 Feb 2024
- 2024HaltingVT: Adaptive Token Halting Transformer for Efficient Video Recognition10 Jan 2024
- 2024LF-ViT: Reducing Spatial Redundancy in Vision Transformer for Efficient Image Recognition8 Jan 2024
12 newest of 63 All 63 (JSON)