mlsanityData to 7 Oct 2026

System-status-aware Adaptive Network for Online Streaming Video Understanding

arXiv 2303.15742 ↗First posted 28 Mar 2023cs.CVcited by 9

Not on the map. Why?

Abstract (arXiv)

Recent years have witnessed great progress in deep neural networks for real-time applications. However, most existing works do not explicitly consider the general case where the device's state and the available resources fluctuate over time, and none of them investigate or address the impact of varying computational resources for online video understanding tasks. This paper proposes a System-status-aware Adaptive Network (SAN) that considers the device's real-time state to provide high-quality predictions with low delay. Usage of our agent's policy improves efficiency and robustness to fluctuations of the system status. On two widely used video understanding tasks, SAN obtains state-of-the-art performance while constantly keeping processing delays low. Moreover, training such an agent on various types of hardware configurations is not easy as the labeled training data might not be available, or can be computationally prohibitive. To address this challenging problem, we propose a Meta Self-supervised Adaptation (MSA) method that adapts the agent's policy to new hardware configurations at test-time, allowing for easy deployment of the model onto other unseen hardware platforms.

Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Builds on 39 of its 59 references that are in the map. First posted on arXiv 28 Mar 2023.

Builds on

39 of 59 references.

References · 39

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,604
  3. 2015Distilling the Knowledge in a Neural Networkcited by 10,076
  4. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  5. 2017MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applicationscited by 3,827
  6. 2015Continuous control with deep reinforcement learningcited by 3,279
  7. 2015Domain-Adversarial Training of Neural Networkscited by 3,023
  8. 2016Asynchronous Methods for Deep Reinforcement Learningcited by 2,218
  9. 2015Convolutional LSTM Network: A Machine Learning Approach for Precipitation Nowcastingcited by 1,428
  10. 2019Deep High-Resolution Representation Learning for Human Pose Estimationcited by 1,339
  11. 2017ShuffleNet: An Extremely Efficient Convolutional Neural Network for Mobile Devicescited by 1,194
  12. 2021EfficientNetV2: Smaller Models and Faster Trainingcited by 1,127
27 more
  1. 2016SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model sizecited by 1,054
  2. 2018TSM: Temporal Shift Module for Efficient Video Understandingcited by 852
  3. 2019Once-for-All: Train One Network and Specialize it for Efficient Deploymentcited by 778
  4. 2018ProxylessNAS: Direct Neural Architecture Search on Target Task and Hardwarecited by 666
  5. 2015BinaryConnect: Training Deep Neural Networks with binary weights during propagationscited by 571
  6. 2018Simple Baselines for Human Pose Estimation and Trackingcited by 502
  7. 2017SkipNet: Learning Dynamic Routing in Convolutional Networkscited by 328
  8. 2019CondConv: Conditionally Parameterized Convolutions for Efficient Inferencecited by 273
  9. 2017Incremental Network Quantization: Towards Lossless CNNs with Low-Precision Weightscited by 211
  10. 2018Task-Agnostic Meta-Learning for Few-shot Learningcited by 151
  11. 2020HAT: Hardware-Aware Transformers for Efficient Natural Language Processingcited by 136
  12. 2020Self-Supervised Policy Adaptation during Deploymentcited by 121
  13. 2018ECO: Efficient Convolutional Network for Online Video Understandingcited by 115
  14. 2020AR-Net: Adaptive Frame Resolution for Efficient Action Recognitioncited by 88
  15. 2013The Optimal Reward Baseline for Gradient-Based Reinforcement Learningcited by 84
  16. 2019LiteEval: A Coarse-to-Fine Framework for Resource Efficient Video Recognitioncited by 65
  17. 2021Manifold Regularized Dynamic Network Pruningcited by 43
  18. 2021Skip-Convolutions for Efficient Video Processingcited by 32
  19. 2019Dynamic Kernel Distillation for Efficient Pose Estimation in Videoscited by 30
  20. 2021ViPNAS: Efficient Video Pose Estimation via Neural Architecture Searchcited by 26
  21. 20202D or not 2D? Adaptive 3D Convolution Selection for Efficient Video Recognitioncited by 22
  22. 2020Intermittent Inference with Nonuniformly Compressed Multi-Exit Neural Network for Energy Harvesting Powered Devicescited by 19
  23. 2022Dynamic Spatio-Temporal Specialization Learning for Fine-Grained Action Recognitioncited by 17
  24. 2021HELP: Hardware-Adaptive Efficient Latency Prediction for NAS via Meta-Learningcited by 16
  25. 2021Generalized Domain Adaptationcited by 10
  26. 2022ERA: Expert Retrieval and Assembly for Early Action Predictioncited by 9
  27. 2019DDLSTM: Dual-Domain LSTM for Cross-Dataset Action Recognitioncited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 3; 2024: 4; 2025: 1; 2026: 1.

9 papers.

  1. 2024Number it: Temporal Grounding Videos like Flipping Mangacited by 65
  2. 2024TR-DETR: Task-Reciprocal Transformer for Joint Moment Retrieval and Highlight Detectioncited by 59
  3. 2023Exploring Incompatible Knowledge Transfer in Few-shot Image Generationcited by 19
  4. 2024Learning to Unlearn for Robust Machine Unlearningcited by 19
  5. 2023Generative Model-based Feature Knowledge Distillation for Action Recognitioncited by 7
  6. 2023Token Boosting for Robust Self-Supervised Visual Transformer Pre-trainingcited by 4
  7. 2024Di2Pose: Discrete Diffusion Model for Occluded 3D Human Pose Estimationcited by 2
  8. 2026Leveraging Text-to-Image Diffusion Models for Unsupervised Visual Object Trackingcited by 1
  9. 2025See, Rank, and Filter: Important Word-Aware Clip Filtering via Scene Understanding for Moment Retrieval and Highlight Detectioncited by 0