# AnalogNets: ML-HW Co-Design of Noise-robust TinyML Models and Always-On Analog Compute-in-Memory Accelerator

arXiv: https://arxiv.org/abs/2111.06503 · HTML: https://mlsanity.com/paper/2111.06503/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 55 references that are in the map.
- First posted on arXiv 10 Nov 2021.

## Builds on (most cited first, 16 of 16)

- [MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications](https://mlsanity.com/paper/1704.04861/) (2017)
- [Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation](https://mlsanity.com/paper/1308.3432/) (2013)
- [Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference](https://mlsanity.com/paper/1712.05877/) (2017)
- [EfficientNetV2: Smaller Models and Faster Training](https://mlsanity.com/paper/2104.00298/) (2021)
- [Speech Commands: A Dataset for Limited-Vocabulary Speech Recognition](https://mlsanity.com/paper/1804.03209/) (2018)
- [MCUNet: Tiny Deep Learning on IoT Devices](https://mlsanity.com/paper/2007.10319/) (2020)
- [Training with Quantization Noise for Extreme Model Compression](https://mlsanity.com/paper/2004.07320/) (2020)
- [Parametric Noise Injection: Trainable Randomness to Improve Deep Neural Network Robustness against Adversarial Attack](https://mlsanity.com/paper/1811.09310/) (2018)
- [Trained Quantization Thresholds for Accurate and Efficient Fixed-Point Inference of Deep Neural Networks](https://mlsanity.com/paper/1903.08066/) (2019)
- [PUMA: A Programmable Ultra-efficient Memristor-based Accelerator for Machine Learning Inference](https://mlsanity.com/paper/1901.10351/) (2019)
- [Regularizing Deep Neural Networks by Noise: Its Interpretation and Optimization](https://mlsanity.com/paper/1710.05179/) (2017)
- [SpArSe: Sparse Architecture Search for CNNs on Resource-Constrained Microcontrollers](https://mlsanity.com/paper/1905.12107/) (2019)
- [Visual Wake Words Dataset](https://mlsanity.com/paper/1906.05721/) (2019)
- [TinyLSTMs: Efficient Neural Speech Enhancement for Hearing Aids](https://mlsanity.com/paper/2005.11138/) (2020)
- [Network insensitivity to parameter noise via adversarial regularization](https://mlsanity.com/paper/2106.05009/) (2021)
- [Training DNN IoT Applications for Deployment On Analog NVM Crossbars](https://mlsanity.com/paper/1910.13850/) (2019)

## Built on by (most cited first, 7 of 7)

- [P2M: A Processing-in-Pixel-in-Memory Paradigm for Resource-Constrained TinyML Applications](https://mlsanity.com/paper/2203.04737/) (2022)
- [A Heterogeneous In-Memory Computing Cluster For Flexible End-to-End Inference of Real-World Deep Neural Networks](https://mlsanity.com/paper/2201.01089/) (2022)
- [NeuroSim V1.5: Improved Software Backbone for Benchmarking Compute-in-Memory Accelerators with Device and Circuit-level Non-idealities](https://mlsanity.com/paper/2505.02314/) (2025)
- [Designing Object Detection Models for TinyML: Foundations, Comparative Analysis, Challenges, and Emerging Solutions](https://mlsanity.com/paper/2508.08352/) (2025)
- [CIMNAS: A Joint Framework for Compute-In-Memory-Aware Neural Architecture Search](https://mlsanity.com/paper/2509.25862/) (2025)
- [Impact of L1 Batch Normalization on Analog Noise Resistant Property of Deep Learning Models](https://mlsanity.com/paper/2205.04886/) (2022)
- [LLM-Guided Neural Architecture Search for Robust Co-Design of Physical Neural Networks](https://mlsanity.com/paper/2606.10294/) (2026)

## Abstract (arXiv)

Always-on TinyML perception tasks in IoT applications require very high energy efficiency. Analog compute-in-memory (CiM) using non-volatile memory (NVM) promises high efficiency and also provides self-contained on-chip model storage. However, analog CiM introduces new practical considerations, including conductance drift, read/write noise, fixed analog-to-digital (ADC) converter gain, etc. These additional constraints must be addressed to achieve models that can be deployed on analog CiM with acceptable accuracy loss. This work describes textitAnalogNets: TinyML models for the popular always-on applications of keyword spotting (KWS) and visual wake words (VWW). The model architectures are specifically designed for analog CiM, and we detail a comprehensive training methodology, to retain accuracy in the face of analog non-idealities, and low-precision data converters at inference time. We also describe AON-CiM, a programmable, minimal-area phase-change memory (PCM) analog CiM accelerator, with a novel layer-serial approach to remove the cost of complex interconnects associated with a fully-pipelined design. We evaluate the AnalogNets on a calibrated simulator, as well as real hardware, and find that accuracy degradation is limited to 0.8%/1.2% after 24 hours of PCM drift (8-bit) for KWS/VWW. AnalogNets running on the 14nm AON-CiM accelerator demonstrate 8.58/4.37 TOPS/W for KWS/VWW workloads using 8-bit activations, respectively, and increasing to 57.39/25.69 TOPS/W with 4-bit activations.