# Real-Time Video Inference on Edge Devices via Adaptive Model Streaming

arXiv: https://arxiv.org/abs/2006.06628 · HTML: https://mlsanity.com/paper/2006.06628/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 11 Jun 2020.

## Built on by (most cited first, 29 of 29)

- [ViLA: Efficient Video-Language Alignment for Video Question Answering](https://mlsanity.com/paper/2312.08367/) (2023)
- [Cloud-Device Collaborative Adaptation to Continual Changing Environments in the Real-world](https://mlsanity.com/paper/2212.00972/) (2022)
- [Shoggoth: Towards Efficient Edge-Cloud Collaborative Real-Time Video Inference via Adaptive Online Learning](https://mlsanity.com/paper/2306.15333/) (2023)
- [Enabling Resource-efficient AIoT System with Cross-level Optimization: A survey](https://mlsanity.com/paper/2309.15467/) (2023)
- [DaCapo: Accelerating Continuous Learning in Autonomous Systems for Video Analytics](https://mlsanity.com/paper/2403.14353/) (2024)
- [Adaptive Stream Processing on Edge Devices through Active Inference](https://mlsanity.com/paper/2409.17937/) (2024)
- [Enhancing Video Analytics Accuracy via Real-time Automated Camera Parameter Tuning](https://mlsanity.com/paper/2107.03964/) (2021)
- [Enabling All In-Edge Deep Learning: A Literature Review](https://mlsanity.com/paper/2204.03326/) (2022)
- [EdgeOL: Efficient in-situ Online Learning on Edge Devices](https://mlsanity.com/paper/2401.16694/) (2024)
- [DeDelayed: Deleting Remote Inference Delay via On-Device Correction](https://mlsanity.com/paper/2510.13714/) (2025)
- [EdgeMA: Model Adaptation System for Real-Time Video Analytics on Edge Devices](https://mlsanity.com/paper/2308.08717/) (2023)
- [On the Query Strategies for Efficient Online Active Distillation](https://mlsanity.com/paper/2309.01612/) (2023)
- [EdgeSync: Faster Edge-model Updating via Adaptive Continuous Learning for Video Data Drift](https://mlsanity.com/paper/2406.03001/) (2024)
- [High-Quality Pseudo-Label Generation Based on Visual Prompt Assisted Cloud Model Update](https://mlsanity.com/paper/2504.00526/) (2025)
- [Real-Time, Low-Latency Surveillance Using Entropy-Based Adaptive Buffering and MobileNetV2 on Edge Devices](https://mlsanity.com/paper/2506.14833/) (2025)
- [LQA: A Lightweight Quantized-Adaptive Framework for Vision-Language Models on the Edge](https://mlsanity.com/paper/2602.07849/) (2026)
- [Design Methodology and Performance Trade-offs Management for Distributed and Compound AI Systems](https://mlsanity.com/paper/2606.14350/) (2026)
- [APT: Adaptive Perceptual quality based camera Tuning using reinforcement learning](https://mlsanity.com/paper/2211.08504/) (2022)
- [Elixir: A system to enhance data quality for multiple analytics on a video stream](https://mlsanity.com/paper/2212.04061/) (2022)
- [Online Resource Allocation for Edge Intelligence with Colocated Model Retraining and Inference](https://mlsanity.com/paper/2405.16029/) (2024)
- [Legilimens: Performant Video Analytics on the System-on-Chip Edge](https://mlsanity.com/paper/2504.21136/) (2025)
- [Responsive DNN Adaptation for Video Analytics against Environment Shift via Hierarchical Mobile-Cloud Collaborations](https://mlsanity.com/paper/2505.00745/) (2025)
- [Catastrophic Forgetting Mitigation via Discrepancy-Weighted Experience Replay](https://mlsanity.com/paper/2507.00042/) (2025)
- [Polymorph: Energy-Efficient Multi-Label Classification for Video Streams on Embedded Devices](https://mlsanity.com/paper/2507.14959/) (2025)
- [EdgeSync: Accelerating Edge-Model Updates for Data Drift through Adaptive Continuous Learning](https://mlsanity.com/paper/2510.21781/) (2025)
- [ECCO: Leveraging Cross-Camera Correlations for Efficient Live Video Continuous Learning](https://mlsanity.com/paper/2512.11727/) (2025)
- [RedunCut: Measurement-Driven Sampling and Accuracy Performance Modeling for Low-Cost Live Video Analytics](https://mlsanity.com/paper/2512.24386/) (2025)
- [A Microservice-Based Platform for Sustainable and Intelligent SLO Fulfilment and Service Management](https://mlsanity.com/paper/2602.12875/) (2026)
- [AdaptAV: Continuous Adaption of Vision Models for Autonomous Vehicles Using Cloud-based Oracle](https://mlsanity.com/paper/2608.28673/) (2026)

## Abstract (arXiv)

Real-time video inference on edge devices like mobile phones and drones is challenging due to the high computation cost of Deep Neural Networks. We present Adaptive Model Streaming (AMS), a new approach to improving performance of efficient lightweight models for video inference on edge devices. AMS uses a remote server to continually train and adapt a small model running on the edge device, boosting its performance on the live video using online knowledge distillation from a large, state-of-the-art model. We discuss the challenges of over-the-network model adaptation for video inference, and present several techniques to reduce communication cost of this approach: avoiding excessive overfitting, updating a small fraction of important model parameters, and adaptive sampling of training frames at edge devices. On the task of video semantic segmentation, our experimental results show 0.4--17.8 percent mean Intersection-over-Union improvement compared to a pre-trained model across several video datasets. Our prototype can perform video segmentation at 30 frames-per-second with 40 milliseconds camera-to-label latency on a Samsung Galaxy S10+ mobile phone, using less than 300 Kbps uplink and downlink bandwidth on the device.