mlsanityData to 14 Oct 2026

Cost-Driven Hardware-Software Co-Optimization of Machine Learning Pipelines

arXiv 2310.07940 ↗First posted 11 Oct 2023cs.LGcited by 0

Not on the map. Why?

Abstract (arXiv)

Researchers have long touted a vision of the future enabled by a proliferation of internet-of-things devices, including smart sensors, homes, and cities. Increasingly, embedding intelligence in such devices involves the use of deep neural networks. However, their storage and processing requirements make them prohibitive for cheap, off-the-shelf platforms. Overcoming those requirements is necessary for enabling widely-applicable smart devices. While many ways of making models smaller and more efficient have been developed, there is a lack of understanding of which ones are best suited for particular scenarios. More importantly for edge platforms, those choices cannot be analyzed in isolation from cost and user experience. In this work, we holistically explore how quantization, model scaling, and multi-modality interact with system components such as memory, sensors, and processors. We perform this hardware/software co-design from the cost, latency, and user-experience perspective, and develop a set of guidelines for optimal system design and model deployment for the most cost-constrained platforms. We demonstrate our approach using an end-to-end, on-device, biometric user authentication system using a 20 ESP-EYE board.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 33 of its 70 references that are in the map. First posted on arXiv 11 Oct 2023.

Builds on

33 of 70 references.

References · 33

  1. 2015Deep Residual Learning for Image Recognitioncited by 52,096
  2. 2017MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applicationscited by 3,845
  3. 2015Learning both Weights and Connections for Efficient Neural Networkscited by 2,031
  4. 2016SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <0.5MB model sizecited by 1,060
  5. 2018VoxCeleb2: Deep Speaker Recognitioncited by 875
  6. 2016XNOR-Net: ImageNet Classification Using Binary Convolutional Neural Networkscited by 849
  7. 2017VoxCeleb: a large-scale speaker identification datasetcited by 732
  8. 2016TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systemscited by 664
  9. 2015BinaryConnect: Training Deep Neural Networks with binary weights during propagationscited by 592
  10. 2016DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradientscited by 569
  11. 2015Deep Learning with Limited Numerical Precisioncited by 376
  12. 2020MCUNet: Tiny Deep Learning on IoT Devicescited by 273
21 more
  1. 2020TensorFlow Lite Micro: Embedded Machine Learning on TinyML Systemscited by 165
  2. 2016FINN: A Framework for Fast, Scalable Binarized Neural Network Inferencecited by 150
  3. 2015Fixed Point Quantization of Deep Convolutional Networkscited by 143
  4. 2021MLPerf Tiny Benchmarkcited by 124
  5. 2018CMSIS-NN: Efficient Neural Network Kernels for Arm Cortex-M CPUscited by 121
  6. 2018ChamNet: Towards Efficient Network Design through Platform-Aware Model Adaptationcited by 76
  7. 2022Killing Two Birds with One Stone:Efficient and Robust Training of Face Recognition CNNs by Partial FCcited by 74
  8. 2018Seeing Voices and Hearing Faces: Cross-modal biometric matchingcited by 60
  9. 2020HAWQV3: Dyadic Neural Network Quantizationcited by 60
  10. 2019DeepShift: Towards Multiplication-Less Neural Networkscited by 54
  11. 2019Memory-Driven Mixed Low Precision Quantization For Enabling Deep Network Inference On Microcontrollerscited by 33
  12. 2018XNOR Neural Engine: a Hardware Accelerator IP for 21.6 fJ/op Binary Neural Network Inferencecited by 30
  13. 2020AutoSpeech: Neural Architecture Search for Speaker Recognitioncited by 17
  14. 2021Differentiable Network Pruning for Microcontrollerscited by 16
  15. 2017BMXNet: An Open-Source Binary Neural Network Implementation Based on MXNetcited by 11
  16. 2017Embedded Binarized Neural Networkscited by 8
  17. 2018Automating Generation of Low Precision Deep Learning Operatorscited by 6
  18. 2018XNORBIN: A 95 TOp/s/W Hardware Accelerator for Binary Convolutional Neural Networkscited by 5
  19. 2017Scaling Binarized Neural Networks on Reconfigurable Logiccited by 4
  20. 2017A GPU-Outperforming FPGA Accelerator Architecture for Binary Convolutional Neural Networkscited by 2
  21. 2017Espresso: Efficient Forward Propagation for BCNNscited by 2

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2022On-Device Training Under 256KB Memory
  2. 2025A Systematic Evaluation of On-Device LLMs: Quantization, Performance, and Resources
  3. 2022Adaptation of MobileNetV2 for Face Detection on Ultra-Low Power Platform
  4. 2025From Tiny Machine Learning to Tiny Deep Learning: A Survey
  5. 2025Neural Network Quantization for Microcontrollers: A Comprehensive Survey of Methods, Platforms, and Applications
  6. 2021Quantization and Deployment of Deep Neural Networks on Microcontrollers
  7. 2026Energy Efficient Software Hardware CoDesign for Machine Learning: From TinyML to Large Language Models
  8. 2022Enable Deep Learning on Mobile Devices: Methods, Systems, and Applications