# MoCA: Memory-Centric, Adaptive Execution for Multi-Tenant Deep Neural Networks

arXiv: https://arxiv.org/abs/2305.05843 · HTML: https://mlsanity.com/paper/2305.05843/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 53 references that are in the map.
- First posted on arXiv 10 May 2023.

## Builds on (most cited first, 19 of 19)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Going Deeper with Convolutions](https://mlsanity.com/paper/1409.4842/) (2014)
- [YOLO9000: Better, Faster, Stronger](https://mlsanity.com/paper/1612.08242/) (2016)
- [SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model size](https://mlsanity.com/paper/1602.07360/) (2016)
- [In-Datacenter Performance Analysis of a Tensor Processing Unit](https://mlsanity.com/paper/1704.04760/) (2017)
- [MLPerf Inference Benchmark](https://mlsanity.com/paper/1911.02549/) (2019)
- [MLPerf Training Benchmark](https://mlsanity.com/paper/1910.01500/) (2019)
- [The Limits and Potentials of Deep Learning for Robotics](https://mlsanity.com/paper/1804.06557/) (2018)
- [Benchmark Analysis of Representative Deep Neural Network Architectures](https://mlsanity.com/paper/1810.00736/) (2018)
- [Deep Residual Learning for Small-Footprint Keyword Spotting](https://mlsanity.com/paper/1710.10361/) (2017)
- [Gemmini: Enabling Systematic Deep-Learning Architecture Evaluation via Full-Stack Integration](https://mlsanity.com/paper/1911.09925/) (2019)
- [Combining Optimal Control and Learning for Visual Navigation in Novel Environments](https://mlsanity.com/paper/1903.02531/) (2019)
- [YOLO-LITE: A Real-Time Object Detection Algorithm Optimized for Non-GPU Computers](https://mlsanity.com/paper/1811.05588/) (2018)
- [PREMA: A Predictive Multi-task Scheduling Algorithm For Preemptible Neural Processing Units](https://mlsanity.com/paper/1909.04548/) (2019)
- [MAGMA: An Optimization Framework for Mapping Multiple DNNs on Multiple Accelerator Cores](https://mlsanity.com/paper/2104.13997/) (2021)
- [VELTAIR: Towards High-Performance Multi-tenant Deep Learning Services via Adaptive Compilation and Scheduling](https://mlsanity.com/paper/2201.06212/) (2022)
- [MAVBench: Micro Aerial Vehicle Benchmarking](https://mlsanity.com/paper/1905.06388/) (2019)
- [Exploring Extended Reality with ILLIXR: A New Playground for Architecture Research](https://mlsanity.com/paper/2004.04643/) (2020)

## Built on by (most cited first, 8 of 8)

- [SCAR: Scheduling Multi-Model AI Workloads on Heterogeneous Multi-Chiplet Module Accelerators](https://mlsanity.com/paper/2405.00790/) (2024)
- [Shared Memory-contention-aware Concurrent DNN Execution for Diversely Heterogeneous System-on-Chips](https://mlsanity.com/paper/2308.05869/) (2023)
- [Deep Reinforcement Learning based Online Scheduling Policy for Deep Neural Network Multi-Tenant Multi-Accelerator Systems](https://mlsanity.com/paper/2404.08950/) (2024)
- [Towards Fair and Firm Real-Time Scheduling in DNN Multi-Tenant Multi-Accelerator Systems via Reinforcement Learning](https://mlsanity.com/paper/2403.00766/) (2024)
- [CaMDN: Enhancing Cache Efficiency for Multi-tenant DNNs on Integrated NPUs](https://mlsanity.com/paper/2505.06625/) (2025)
- [Orchestrating Multimodal DNN Workloads in Wireless Neural Processing](https://mlsanity.com/paper/2603.02109/) (2026)
- [R2E-VID: Two-Stage Robust Routing via Temporal Gating for Elastic Edge-Cloud Video Inference](https://mlsanity.com/paper/2604.09681/) (2026)
- [Terastal: Layer-Variant-based Scheduling for Real-Time Multi-DNN Workloads on Heterogeneous Accelerators](https://mlsanity.com/paper/2606.06818/) (2026)

## Abstract (arXiv)

Driven by the wide adoption of deep neural networks (DNNs) across different application domains, multi-tenancy execution, where multiple DNNs are deployed simultaneously on the same hardware, has been proposed to satisfy the latency requirements of different applications while improving the overall system utilization. However, multi-tenancy execution could lead to undesired system-level resource contention, causing quality-of-service (QoS) degradation for latency-critical applications. To address this challenge, we propose MoCA, an adaptive multi-tenancy system for DNN accelerators. Unlike existing solutions that focus on compute resource partition, MoCA dynamically manages shared memory resources of co-located applications to meet their QoS targets. Specifically, MoCA leverages the regularities in both DNN operators and accelerators to dynamically modulate memory access rates based on their latency targets and user-defined priorities so that co-located applications get the resources they demand without significantly starving their co-runners. We demonstrate that MoCA improves the satisfaction rate of the service level agreement (SLA) up to 3.9x (1.8x average), system throughput by 2.3x (1.7x average), and fairness by 1.3x (1.2x average), compared to prior work.