mlsanityData to 14 Oct 2026

CaMDN: Enhancing Cache Efficiency for Multi-tenant DNNs on Integrated NPUs

arXiv 2505.06625 ↗First posted 10 May 2025cs.AR cs.AI cs.OScited by 0

Not on the map. Why?

Abstract (arXiv)

With the rapid development of DNN applications, multi-tenant execution, where multiple DNNs are co-located on a single SoC, is becoming a prevailing trend. Although many methods are proposed in prior works to improve multi-tenant performance, the impact of shared cache is not well studied. This paper proposes CaMDN, an architecture-scheduling co-design to enhance cache efficiency for multi-tenant DNNs on integrated NPUs. Specifically, a lightweight architecture is proposed to support model-exclusive, NPU-controlled regions inside shared cache to eliminate unexpected cache contention. Moreover, a cache scheduling method is proposed to improve shared cache utilization. In particular, it includes a cache-aware mapping method for adaptability to the varying available cache capacity and a dynamic allocation algorithm to adjust the usage among co-located DNNs at runtime. Compared to prior works, CaMDN reduces the memory access by 33.4% on average and achieves a model speedup of up to 2.56× (1.88× on average).

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 18 of its 39 references that are in the map. First posted on arXiv 10 May 2025.

Builds on

18 of 39 references.

References · 18

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  4. 2019EfficientNet: Rethinking Model Scaling for Convolutional Neural Networkscited by 6,442
  5. 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,795
  6. 2020wav2vec 2.0: A Framework for Self-Supervised Learning of Speech Representationscited by 3,905
  7. 2018PointPillars: Fast Encoders for Object Detection from Point Cloudscited by 1,715
  8. 2016Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translationcited by 1,475
  9. 2019Gemmini: Enabling Systematic Deep-Learning Architecture Evaluation via Full-Stack Integrationcited by 63
  10. 2019PREMA: A Predictive Multi-task Scheduling Algorithm For Preemptible Neural Processing Unitscited by 30
  11. 2021CoSA: Scheduling by Constrained Optimization for Spatial Acceleratorscited by 22
  12. 2021MAGMA: An Optimization Framework for Mapping Multiple DNNs on Multiple Accelerator Corescited by 15
6 more
  1. 2022VELTAIR: Towards High-Performance Multi-tenant Deep Learning Services via Adaptive Compilation and Schedulingcited by 14
  2. 2023OmniBoost: Boosting Throughput of Heterogeneous Embedded Devices under Multi-DNN Workloadcited by 12
  3. 2022DREAM: A Dynamic Scheduler for Dynamic Real-time Multi-model ML Workloadscited by 8
  4. 2023MoCA: Memory-Centric, Adaptive Execution for Multi-Tenant Deep Neural Networkscited by 8
  5. 2024Deep Reinforcement Learning based Online Scheduling Policy for Deep Neural Network Multi-Tenant Multi-Accelerator Systemscited by 4
  6. 2021Cohmeleon: Learning-Based Orchestration of Accelerator Coherence in Heterogeneous SoCscited by 1

Built on by

Not cited yet.