mlsanityData to 7 Oct 2026

OmniBoost: Boosting Throughput of Heterogeneous Embedded Devices under Multi-DNN Workload

arXiv 2307.03290 ↗First posted 6 Jul 2023cs.LG cs.PFcited by 12

Not on the map. Why?

Abstract (arXiv)

Modern Deep Neural Networks (DNNs) exhibit profound efficiency and accuracy properties. This has introduced application workloads that comprise of multiple DNN applications, raising new challenges regarding workload distribution. Equipped with a diverse set of accelerators, newer embedded system present architectural heterogeneity, which current run-time controllers are unable to fully utilize. To enable high throughput in multi-DNN workloads, such a controller is ought to explore hundreds of thousands of possible solutions to exploit the underlying heterogeneity. In this paper, we propose OmniBoost, a lightweight and extensible multi-DNN manager for heterogeneous embedded devices. We leverage stochastic space exploration and we combine it with a highly accurate performance estimator to observe a x4.6 average throughput boost compared to other state-of-the-art methods. The evaluation was performed on the HiKey970 development board.

Key facts. Cited by 12 papers in the map (data through 7 Oct 2026). Builds on 15 of its 30 references that are in the map. First posted on arXiv 6 Jul 2023.

Builds on

15 of 30 references.

References · 15

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  3. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  4. 2015Rethinking the Inception Architecture for Computer Visioncited by 5,451
  5. 2017MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applicationscited by 3,827
  6. 2013Distributed Representations of Words and Phrases and their Compositionalitycited by 3,462
  7. 2016Gaussian Error Linear Units (GELUs)cited by 3,431
  8. 2016OpenAI Gymcited by 1,684
  9. 2016Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learningcited by 1,509
  10. 2016SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model sizecited by 1,054
  11. 2021Monte Carlo Tree Search: A Review of Recent Modifications and Applicationscited by 158
  12. 2019High-Throughput CNN Inference on Embedded ARM big.LITTLE Multi-Core Processorscited by 19
3 more
  1. 2021A Survey and Empirical Evaluation of Parallel Deep Learning Frameworkscited by 7
  2. 2015CNNdroid: GPU-Accelerated Execution of Trained Deep Convolutional Neural Networks on Androidcited by 4
  3. 2022Real-Time Scheduling of Machine Learning Operations on Heterogeneous Neuromorphic SoCcited by 2

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 5; 2025: 4; 2026: 2.

12 papers.

  1. 2024An LLM-Tool Compiler for Fused Parallel Function Callingcited by 14
  2. 2023Shared Memory-contention-aware Concurrent DNN Execution for Diversely Heterogeneous System-on-Chipscited by 5
  3. 2024HiDP: Hierarchical DNN Partitioning for Distributed Inference on Heterogeneous Edge Platformscited by 4
  4. 2024RankMap: Priority-Aware Multi-DNN Manager for Heterogeneous Embedded Devicescited by 4
  5. 2025Ecomap: Sustainability-Driven Optimization of Multi-Tenant DNN Execution on Edge Serverscited by 2
  6. 2024CARIn: Constraint-Aware and Responsive Inference on Heterogeneous Devices for Single- and Multi-DNN Workloadscited by 1
  7. 2024Optimizing DNN Inference on Multi-Accelerator SoCs at Training-timecited by 0
  8. 2025CaMDN: Enhancing Cache Efficiency for Multi-tenant DNNs on Integrated NPUscited by 0
  9. 2025Twill: Scheduling Compound AI Systems on Heterogeneous Mobile Edge Platformscited by 0
  10. 2025Puzzle: Scheduling Multiple Deep Learning Models on Mobile Device with Heterogeneous Processorscited by 0
  11. 2026Multi-DNN Inference of Sparse Models on Edge SoCscited by 0
  12. 2026TAPAS: Throughput-adaptive Perception for Autonomous Systemscited by 0