# Examining the Role and Limits of Batchnorm Optimization to Mitigate Diverse Hardware-noise in In-memory Computing

arXiv: https://arxiv.org/abs/2305.18416 · HTML: https://mlsanity.com/paper/2305.18416/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 11 of its 26 references that are in the map.
- First posted on arXiv 28 May 2023.

## Builds on (most cited first, 11 of 11)

- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [Rethinking the Inception Architecture for Computer Vision](https://mlsanity.com/paper/1512.00567/) (2015)
- [Hardware-aware training for large-scale and diverse deep learning inference workloads using in-memory computing-based accelerators](https://mlsanity.com/paper/2302.08469/) (2023)
- [A flexible and fast PyTorch toolkit for simulating training and inference on analog crossbar arrays](https://mlsanity.com/paper/2104.02184/) (2021)
- [DNN+NeuroSim V2.0: An End-to-End Benchmarking Framework for Compute-in-Memory Accelerators for On-chip Training](https://mlsanity.com/paper/2003.06471/) (2020)
- [RxNN: A Framework for Evaluating Deep Neural Networks on Resistive Crossbars](https://mlsanity.com/paper/1809.00072/) (2018)
- [Multiscale Co-Design Analysis of Energy, Latency, Area, and Accuracy of a ReRAM Analog Neural Training Accelerator](https://mlsanity.com/paper/1707.09952/) (2017)
- [TxSim:Modeling Training of Deep Neural Networks on Resistive Crossbar Systems](https://mlsanity.com/paper/2002.11151/) (2020)
- [Examining the Robustness of Spiking Neural Networks on Non-ideal Memristive Crossbars](https://mlsanity.com/paper/2206.09599/) (2022)
- [Mitigate Parasitic Resistance in Resistive Crossbar-based Convolutional Neural Networks](https://mlsanity.com/paper/1912.08716/) (2019)
- [NAX: Co-Designing Neural Network and Hardware Architecture for Memristive Xbar based Computing Systems](https://mlsanity.com/paper/2106.12125/) (2021)

## Built on by (most cited first, 5 of 5)

- [ClipFormer: Key-Value Clipping of Transformers on Memristive Crossbars for Write Noise Mitigation](https://mlsanity.com/paper/2402.02586/) (2024)
- [Enabling Energy-Efficient Deployment of Large Language Models on Memristor Crossbar: A Synergy of Large and Small](https://mlsanity.com/paper/2410.15977/) (2024)
- [TReX- Reusing Vision Transformer's Attention for Efficient Xbar-based Computing](https://mlsanity.com/paper/2408.12742/) (2024)
- [When In-memory Computing Meets Spiking Neural Networks -- A Perspective on Device-Circuit-System-and-Algorithm Co-design](https://mlsanity.com/paper/2408.12767/) (2024)
- [ThRIve: Thermally Robust CNN Inference via Low-Rank Adaptation in Heterogeneous PIM Architectures](https://mlsanity.com/paper/2607.17091/) (2026)

## Abstract (arXiv)

In-Memory Computing (IMC) platforms such as analog crossbars are gaining focus as they facilitate the acceleration of low-precision Deep Neural Networks (DNNs) with high area- & compute-efficiencies. However, the intrinsic non-idealities in crossbars, which are often non-deterministic and non-linear, degrade the performance of the deployed DNNs. In addition to quantization errors, most frequently encountered non-idealities during inference include crossbar circuit-level parasitic resistances and device-level non-idealities such as stochastic read noise and temporal drift. In this work, our goal is to closely examine the distortions caused by these non-idealities on the dot-product operations in analog crossbars and explore the feasibility of a nearly training-less solution via crossbar-aware fine-tuning of batchnorm parameters in real-time to mitigate the impact of the non-idealities. This enables reduction in hardware costs in terms of memory and training energy for IMC noise-aware retraining of the DNN weights on crossbars.