# CiMNet: Towards Joint Optimization for DNN Architecture and Configuration for Compute-In-Memory Hardware

arXiv: https://arxiv.org/abs/2402.11780 · HTML: https://mlsanity.com/paper/2402.11780/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 18 of its 36 references that are in the map.
- First posted on arXiv 19 Feb 2024.

## Builds on (most cited first, 18 of 18)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [LLaMA: Open and Efficient Foundation Language Models](https://mlsanity.com/paper/2302.13971/) (2023)
- [Neural Architecture Search with Reinforcement Learning](https://mlsanity.com/paper/1611.01578/) (2016)
- [Learning Transferable Architectures for Scalable Image Recognition](https://mlsanity.com/paper/1707.07012/) (2017)
- [Once-for-All: Train One Network and Specialize it for Efficient Deployment](https://mlsanity.com/paper/1908.09791/) (2019)
- [In-Datacenter Performance Analysis of a Tensor Processing Unit](https://mlsanity.com/paper/1704.04760/) (2017)
- [FBNet: Hardware-Aware Efficient ConvNet Design via Differentiable Neural Architecture Search](https://mlsanity.com/paper/1812.03443/) (2018)
- [MCUNet: Tiny Deep Learning on IoT Devices](https://mlsanity.com/paper/2007.10319/) (2020)
- [HAT: Hardware-Aware Transformers for Efficient Natural Language Processing](https://mlsanity.com/paper/2005.14187/) (2020)
- [Single-Path NAS: Designing Hardware-Efficient ConvNets in less than 4 Hours](https://mlsanity.com/paper/1904.02877/) (2019)
- [FPGA/DNN Co-Design: An Efficient Design Methodology for IoT Intelligence on the Edge](https://mlsanity.com/paper/1904.04421/) (2019)
- [Path-Level Network Transformation for Efficient Architecture Search](https://mlsanity.com/paper/1806.02639/) (2018)
- [Accuracy vs. Efficiency: Achieving Both through FPGA-Implementation Aware Neural Architecture Search](https://mlsanity.com/paper/1901.11211/) (2019)
- [A Tunable Robust Pruning Framework Through Dynamic Network Rewiring of DNNs](https://mlsanity.com/paper/2011.03083/) (2020)
- [A Hardware-Aware Framework for Accelerating Neural Architecture Search Across Modalities](https://mlsanity.com/paper/2205.10358/) (2022)
- [Near-Memory Computing: Past, Present, and Future](https://mlsanity.com/paper/1908.02640/) (2019)
- [Making Models Shallow Again: Jointly Learning to Reduce Non-Linearity and Depth for Latency-Efficient Private Inference](https://mlsanity.com/paper/2304.13274/) (2023)
- [InstaTune: Instantaneous Neural Architecture Search During Fine-Tuning](https://mlsanity.com/paper/2308.15609/) (2023)

## Abstract (arXiv)

With the recent growth in demand for large-scale deep neural networks, compute in-memory (CiM) has come up as a prominent solution to alleviate bandwidth and on-chip interconnect bottlenecks that constrain Von-Neuman architectures. However, the construction of CiM hardware poses a challenge as any specific memory hierarchy in terms of cache sizes and memory bandwidth at different interfaces may not be ideally matched to any neural network's attributes such as tensor dimension and arithmetic intensity, thus leading to suboptimal and under-performing systems. Despite the success of neural architecture search (NAS) techniques in yielding efficient sub-networks for a given hardware metric budget (e.g., DNN execution time or latency), it assumes the hardware configuration to be frozen, often yielding sub-optimal sub-networks for a given budget. In this paper, we present CiMNet, a framework that jointly searches for optimal sub-networks and hardware configurations for CiM architectures creating a Pareto optimal frontier of downstream task accuracy and execution metrics (e.g., latency). The proposed framework can comprehend the complex interplay between a sub-network's performance and the CiM hardware configuration choices including bandwidth, processing element size, and memory size. Exhaustive experiments on different model architectures from both CNN and Transformer families demonstrate the efficacy of the CiMNet in finding co-optimized sub-networks and CiM hardware configurations. Specifically, for similar ImageNet classification accuracy as baseline ViT-B, optimizing only the model architecture increases performance (or reduces workload execution time) by 1.7x while optimizing for both the model architecture and hardware configuration increases it by 3.1x.