# Shared Memory-contention-aware Concurrent DNN Execution for Diversely Heterogeneous System-on-Chips

arXiv: https://arxiv.org/abs/2308.05869 · HTML: https://mlsanity.com/paper/2308.05869/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 26 of its 74 references that are in the map.
- First posted on arXiv 10 Aug 2023.

## Builds on (most cited first, 26 of 26)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [Going Deeper with Convolutions](https://mlsanity.com/paper/1409.4842/) (2014)
- [Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning](https://mlsanity.com/paper/1602.07261/) (2016)
- [A Survey of Deep Learning Techniques for Autonomous Driving](https://mlsanity.com/paper/1910.07738/) (2019)
- [Caffe: Convolutional Architecture for Fast Feature Embedding](https://mlsanity.com/paper/1408.5093/) (2014)
- [DNNFusion: Accelerating Deep Neural Networks Execution with Advanced Operator Fusion](https://mlsanity.com/paper/2108.13342/) (2021)
- [Heterogeneity-Aware Cluster Scheduling Policies for Deep Learning Workloads](https://mlsanity.com/paper/2008.09213/) (2020)
- [HetPipe: Enabling Large DNN Training on (Whimpy) Heterogeneous GPU Clusters through Integration of Pipelined Model Parallelism and Data Parallelism](https://mlsanity.com/paper/2005.14038/) (2020)
- [Mind Mappings: Enabling Efficient Algorithm-Accelerator Mapping Space Search](https://mlsanity.com/paper/2103.01489/) (2021)
- [Google Neural Network Models for Edge Devices: Analyzing and Mitigating Machine Learning Inference Bottlenecks](https://mlsanity.com/paper/2109.14320/) (2021)
- [A Full-Stack Search Technique for Domain Optimized Deep Learning Accelerators](https://mlsanity.com/paper/2105.12842/) (2021)
- [MAGMA: An Optimization Framework for Mapping Multiple DNNs on Multiple Accelerator Cores](https://mlsanity.com/paper/2104.13997/) (2021)
- [OmniBoost: Boosting Throughput of Heterogeneous Embedded Devices under Multi-DNN Workload](https://mlsanity.com/paper/2307.03290/) (2023)
- [MoCA: Memory-Centric, Adaptive Execution for Multi-Tenant Deep Neural Networks](https://mlsanity.com/paper/2305.05843/) (2023)
- [QoS-Aware Placement of Deep Learning Services on the Edge with Multiple Service Implementations](https://mlsanity.com/paper/2104.15094/) (2021)
- [H2H: Heterogeneous Model to Heterogeneous System Mapping with Computation and Communication Awareness](https://mlsanity.com/paper/2204.13852/) (2022)
- [A Computational-Graph Partitioning Method for Training Memory-Constrained DNNs](https://mlsanity.com/paper/2008.08636/) (2020)
- [RoboRun: A Robot Runtime to Exploit Spatial Heterogeneity](https://mlsanity.com/paper/2108.13354/) (2021)
- [Roofline Model for UAVs: A Bottleneck Analysis Tool for Onboard Compute Characterization of Autonomous Unmanned Aerial Vehicles](https://mlsanity.com/paper/2204.10898/) (2022)
- [Zhuyi: Perception Processing Rate Estimation for Safety in Autonomous Vehicles](https://mlsanity.com/paper/2205.03347/) (2022)
- [A Many-ported and Shared Memory Architecture for High-Performance ADAS SoCs](https://mlsanity.com/paper/2209.05731/) (2022)

## Built on by (most cited first, 5 of 5)

- [Ecomap: Sustainability-Driven Optimization of Multi-Tenant DNN Execution on Edge Servers](https://mlsanity.com/paper/2503.04148/) (2025)
- [Optimizing DNN Inference on Multi-Accelerator SoCs at Training-time](https://mlsanity.com/paper/2409.18566/) (2024)
- [Twill: Scheduling Compound AI Systems on Heterogeneous Mobile Edge Platforms](https://mlsanity.com/paper/2507.00491/) (2025)
- [Covariance-Guided Resource Adaptive Learning for Efficient Edge Inference](https://mlsanity.com/paper/2603.14577/) (2026)
- [SelectInfer: Selective Neuron Loading and Computation for On-Device LLMs](https://mlsanity.com/paper/2607.18081/) (2026)

## Abstract (arXiv)

Two distinguishing features of state-of-the-art mobile and autonomous systems are 1) there are often multiple workloads, mainly deep neural network (DNN) inference, running concurrently and continuously; and 2) they operate on shared memory system-on-chips (SoC) that embed heterogeneous accelerators tailored for specific operations. State-of-the-art lacks efficient performance and resource management techniques necessary to either maximize total system throughput or minimize end-to-end workload latency. In this work, we propose HaX-CoNN, a novel scheme that characterizes and maps layers in concurrently executing DNN inference workloads to a diverse set of accelerators within a SoC. Our scheme uniquely takes per-layer execution characteristics, shared memory (SM) contention, and inter-accelerator transitions into account to find optimal schedules. We evaluate HaX-CoNN on NVIDIA Orin, NVIDIA Xavier, and Qualcomm Snapdragon 865 SoCs. Our experimental results indicate that HaX-CoNN minimizes memory contention by up to 45% and can improve latency and total throughput by up to 32% and 29%, respectively, compared to the state-of-the-art approaches.