# DeepRT: A Soft Real Time Scheduler for Computer Vision Applications on the Edge

arXiv: https://arxiv.org/abs/2105.01803 · HTML: https://mlsanity.com/paper/2105.01803/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 48 references that are in the map.
- First posted on arXiv 5 May 2021.

## Builds on (most cited first, 8 of 8)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- [MobileNetV2: Inverted Residuals and Linear Bottlenecks](https://mlsanity.com/paper/1801.04381/) (2018)
- [End to End Learning for Self-Driving Cars](https://mlsanity.com/paper/1604.07316/) (2016)
- [Clipper: A Low-Latency Online Prediction Serving System](https://mlsanity.com/paper/1612.03079/) (2016)
- [TensorFlow-Serving: Flexible, High-Performance ML Serving](https://mlsanity.com/paper/1712.06139/) (2017)
- [Latency and Throughput Characterization of Convolutional Neural Networks for Mobile Computer Vision](https://mlsanity.com/paper/1803.09492/) (2018)

## Built on by (most cited first, 6 of 6)

- [Edge Video Analytics: A Survey on Applications, Systems and Enabling Techniques](https://mlsanity.com/paper/2211.15751/) (2022)
- [PartialLoading: User Scheduling and Bandwidth Allocation for Parameter-sharing Edge Inference](https://mlsanity.com/paper/2503.22982/) (2025)
- [BCEdge: SLO-Aware DNN Inference Services with Adaptive Batching on Edge Platforms](https://mlsanity.com/paper/2305.01519/) (2023)
- [TF2AIF: Facilitating development and deployment of accelerated AI models on the cloud-edge continuum](https://mlsanity.com/paper/2404.13715/) (2024)
- [DMVC: Multi-Camera Video Compression Network aimed at Improving Deep Learning Accuracy](https://mlsanity.com/paper/2410.18400/) (2024)
- [Sensor Calibration Model Balancing Accuracy, Real-time, and Efficiency](https://mlsanity.com/paper/2511.06715/) (2025)

## Abstract (arXiv)

The ubiquity of smartphone cameras and IoT cameras, together with the recent boom of deep learning and deep neural networks, proliferate various computer vision driven mobile and IoT applications deployed on the edge. This paper focuses on applications which make soft real time requests to perform inference on their data - they desire prompt responses within designated deadlines, but occasional deadline misses are acceptable. Supporting soft real time applications on a multi-tenant edge server is not easy, since the requests sharing the limited GPU computing resources of an edge server interfere with each other. In order to tackle this problem, we comprehensively evaluate how latency and throughput respond to different GPU execution plans. Based on this analysis, we propose a GPU scheduler, DeepRT, which provides latency guarantee to the requests while maintaining high overall system throughput. The key component of DeepRT, DisBatcher, batches data from different requests as much as possible while it is proven to provide latency guarantee for requests admitted by an Admission Control Module. DeepRT also includes an Adaptation Module which tackles overruns. Our evaluation results show that DeepRT outperforms state-of-the-art works in terms of the number of deadline misses and throughput.