# A Heterogeneous In-Memory Computing Cluster For Flexible End-to-End Inference of Real-World Deep Neural Networks

arXiv: https://arxiv.org/abs/2201.01089 · HTML: https://mlsanity.com/paper/2201.01089/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 42 references that are in the map.
- First posted on arXiv 4 Jan 2022.

## Builds on (most cited first, 9 of 9)

- [MobileNetV2: Inverted Residuals and Linear Bottlenecks](https://mlsanity.com/paper/1801.04381/) (2018)
- [Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference](https://mlsanity.com/paper/1712.05877/) (2017)
- [DORY: Automatic End-to-End Deployment of Real-World DNNs on Low-Cost IoT MCUs](https://mlsanity.com/paper/2008.07127/) (2020)
- [PULP-NN: Accelerating Quantized Neural Networks on Parallel Ultra-Low-Power RISC-V Processors](https://mlsanity.com/paper/1908.11263/) (2019)
- [Fully Onboard AI-powered Human-Drone Pose Estimation on Ultra-low Power Autonomous Flying Nano-UAVs](https://mlsanity.com/paper/2103.10873/) (2021)
- [Vega: A 10-Core SoC for IoT End-Nodes with DNN Acceleration and Cognitive Wake-Up From MRAM-Based State-Retentive Sleep Mode](https://mlsanity.com/paper/2110.09101/) (2021)
- [A near-threshold RISC-V core with DSP extensions for scalable IoT Endpoint Devices](https://mlsanity.com/paper/1608.08376/) (2016)
- [AnalogNets: ML-HW Co-Design of Noise-robust TinyML Models and Always-On Analog Compute-in-Memory Accelerator](https://mlsanity.com/paper/2111.06503/) (2021)
- [End-to-end 100-TOPS/W Inference With Analog In-Memory Computing: Are We There Yet?](https://mlsanity.com/paper/2109.01404/) (2021)

## Built on by (most cited first, 2 of 2)

- [DDC-PIM: Efficient Algorithm/Architecture Co-design for Doubling Data Capacity of SRAM-based Processing-In-Memory](https://mlsanity.com/paper/2310.20424/) (2023)
- [Zero-shot Classification using Hyperdimensional Computing](https://mlsanity.com/paper/2401.16876/) (2024)

## Abstract (arXiv)

Deployment of modern TinyML tasks on small battery-constrained IoT devices requires high computational energy efficiency. Analog In-Memory Computing (IMC) using non-volatile memory (NVM) promises major efficiency improvements in deep neural network (DNN) inference and serves as on-chip memory storage for DNN weights. However, IMC's functional flexibility limitations and their impact on performance, energy, and area efficiency are not yet fully understood at the system level. To target practical end-to-end IoT applications, IMC arrays must be enclosed in heterogeneous programmable systems, introducing new system-level challenges which we aim at addressing in this work. We present a heterogeneous tightly-coupled clustered architecture integrating 8 RISC-V cores, an in-memory computing accelerator (IMA), and digital accelerators. We benchmark the system on a highly heterogeneous workload such as the Bottleneck layer from a MobileNetV2, showing 11.5x performance and 9.5x energy efficiency improvements, compared to highly optimized parallel execution on the cores. Furthermore, we explore the requirements for end-to-end inference of a full mobile-grade DNN (MobileNetV2) in terms of IMC array resources, by scaling up our heterogeneous architecture to a multi-array accelerator. Our results show that our solution, on the end-to-end inference of the MobileNetV2, is one order of magnitude better in terms of execution latency than existing programmable architectures and two orders of magnitude better than state-of-the-art heterogeneous solutions integrating in-memory computing analog cores.