# Model-driven Cluster Resource Management for AI Workloads in Edge Clouds

arXiv: https://arxiv.org/abs/2201.07312 · HTML: https://mlsanity.com/paper/2201.07312/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Jan 2022.

## Built on by (most cited first, 6 of 6)

- [Empowering Edge Intelligence: A Comprehensive Survey on On-Device AI Models](https://mlsanity.com/paper/2503.06027/) (2025)
- [Offline Energy-Optimal LLM Serving: Workload-Based Energy Models for LLM Inference on Heterogeneous Systems](https://mlsanity.com/paper/2407.04014/) (2024)
- [Uncertainty Estimation in Multi-Agent Distributed Learning for AI-Enabled Edge Devices](https://mlsanity.com/paper/2403.09141/) (2024)
- [Edge AI Collaborative Learning: Bayesian Approaches to Uncertainty Estimation](https://mlsanity.com/paper/2410.08651/) (2024)
- [MEL: Multi-level Ensemble Learning for Resource-Constrained Environments](https://mlsanity.com/paper/2506.20094/) (2025)
- [FMplex: Model Virtualization for Serving Extensible Foundation Models](https://mlsanity.com/paper/2606.09643/) (2026)

## Abstract (arXiv)

Since emerging edge applications such as Internet of Things (IoT) analytics and augmented reality have tight latency constraints, hardware AI accelerators have been recently proposed to speed up deep neural network (DNN) inference run by these applications. Resource-constrained edge servers and accelerators tend to be multiplexed across multiple IoT applications, introducing the potential for performance interference between latency-sensitive workloads. In this paper, we design analytic models to capture the performance of DNN inference workloads on shared edge accelerators, such as GPU and edgeTPU, under different multiplexing and concurrency behaviors. After validating our models using extensive experiments, we use them to design various cluster resource management algorithms to intelligently manage multiple applications on edge accelerators while respecting their latency constraints. We implement a prototype of our system in Kubernetes and show that our system can host 2.3X more DNN applications in heterogeneous multi-tenant edge clusters with no latency violations when compared to traditional knapsack hosting algorithms.