# FORMS: Fine-grained Polarized ReRAM-based In-situ Computation for Mixed-signal DNN Accelerator

arXiv: https://arxiv.org/abs/2106.09144 · HTML: https://mlsanity.com/paper/2106.09144/

## Key facts

- Cited by 12 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 84 references that are in the map.
- First posted on arXiv 16 Jun 2021.

## Builds on (most cited first, 30 of 34)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift](https://mlsanity.com/paper/1502.03167/) (2015)
- [You Only Look Once: Unified, Real-Time Object Detection](https://mlsanity.com/paper/1506.02640/) (2015)
- [Learning both Weights and Connections for Efficient Neural Networks](https://mlsanity.com/paper/1506.02626/) (2015)
- [Learning Efficient Convolutional Networks through Network Slimming](https://mlsanity.com/paper/1708.06519/) (2017)
- [In-Datacenter Performance Analysis of a Tensor Processing Unit](https://mlsanity.com/paper/1704.04760/) (2017)
- [Learning Structured Sparsity in Deep Neural Networks](https://mlsanity.com/paper/1608.03665/) (2016)
- [AMC: AutoML for Model Compression and Acceleration on Mobile Devices](https://mlsanity.com/paper/1802.03494/) (2018)
- [Filter Pruning via Geometric Median for Deep Convolutional Neural Networks Acceleration](https://mlsanity.com/paper/1811.00250/) (2018)
- [Quantized Convolutional Neural Networks for Mobile Devices](https://mlsanity.com/paper/1512.06473/) (2015)
- [Discrimination-aware Channel Pruning for Deep Neural Networks](https://mlsanity.com/paper/1810.11809/) (2018)
- [A Systematic DNN Weight Pruning Framework using Alternating Direction Method of Multipliers](https://mlsanity.com/paper/1804.03294/) (2018)
- [Learning to Track at 100 FPS with Deep Regression Networks](https://mlsanity.com/paper/1604.01802/) (2016)
- [Convergence Analysis of Alternating Direction Method of Multipliers for a Family of Nonconvex Problems](https://mlsanity.com/paper/1410.1390/) (2014)
- [PUMA: A Programmable Ultra-efficient Memristor-based Accelerator for Machine Learning Inference](https://mlsanity.com/paper/1901.10351/) (2019)
- [CirCNN: Accelerating and Compressing Deep Neural Networks Using Block-CirculantWeight Matrices](https://mlsanity.com/paper/1708.08917/) (2017)
- [ADMM-NN: An Algorithm-Hardware Co-Design Framework of DNNs Using Alternating Direction Method of Multipliers](https://mlsanity.com/paper/1812.11677/) (2018)
- [Non-Structured DNN Weight Pruning -- Is It Beneficial in Any Platform?](https://mlsanity.com/paper/1907.02124/) (2019)
- [YOLObile: Real-Time Object Detection on Mobile Devices via Compression-Compilation Co-Design](https://mlsanity.com/paper/2009.05697/) (2020)
- [Zeroth-Order Online Alternating Direction Method of Multipliers: Convergence Analysis and Applications](https://mlsanity.com/paper/1710.07804/) (2017)
- [Multiscale Co-Design Analysis of Energy, Latency, Area, and Accuracy of a ReRAM Analog Neural Training Accelerator](https://mlsanity.com/paper/1707.09952/) (2017)
- [ResNet Can Be Pruned 60x: Introducing Network Purification and Unused Path Removal (P-RM) after Weight Pruning](https://mlsanity.com/paper/1905.00136/) (2019)
- [An Ultra-Efficient Memristor-Based DNN Framework with Structured Weight Pruning and Quantization Using ADMM](https://mlsanity.com/paper/1908.11691/) (2019)
- [TIMELY: Pushing Data Movements and Interfaces in PIM Accelerators Towards Local and in Time Domain](https://mlsanity.com/paper/2005.01206/) (2020)
- [Basic Performance Measurements of the Intel Optane DC Persistent Memory Module](https://mlsanity.com/paper/1903.05714/) (2019)
- [Tiny but Accurate: A Pruned, Quantized and Optimized Memristor Crossbar Framework for Ultra Efficient DNN Implementation](https://mlsanity.com/paper/1908.10017/) (2019)
- [SS-Auto: A Single-Shot, Automatic Structured Weight Pruning Framework of DNNs with Ultra-High Efficiency](https://mlsanity.com/paper/2001.08839/) (2020)
- [Mixed-Signal Charge-Domain Acceleration of Deep Neural networks through Interleaved Bit-Partitioned Arithmetic](https://mlsanity.com/paper/1906.11915/) (2019)
- [Newton: Gravitating Towards the Physical Limits of Crossbar Acceleration](https://mlsanity.com/paper/1803.06913/) (2018)
- [Towards Ultra-High Performance and Energy Efficiency of Deep Learning Systems: An Algorithm-Hardware Co-Optimization Framework](https://mlsanity.com/paper/1802.06402/) (2018)

## Built on by (most cited first, 12 of 12)

- [ANT: Exploiting Adaptive Numerical Data Type for Low-bit Deep Neural Network Quantization](https://mlsanity.com/paper/2208.14286/) (2022)
- [Sparse Attention Acceleration with Synergistic In-Memory Pruning and On-Chip Recomputation](https://mlsanity.com/paper/2209.00606/) (2022)
- [On the Accuracy of Analog Neural Network Inference Accelerators](https://mlsanity.com/paper/2109.01262/) (2021)
- [RoRA: Efficient Fine-Tuning of LLM with Reliability Optimization for Rank Adaptation](https://mlsanity.com/paper/2501.04315/) (2025)
- [Automatic Mapping of the Best-Suited DNN Pruning Schemes for Real-Time Mobile Acceleration](https://mlsanity.com/paper/2111.11581/) (2021)
- [CIM-MLC: A Multi-level Compilation Stack for Computing-In-Memory Accelerators](https://mlsanity.com/paper/2401.12428/) (2024)
- [Zero-Space Cost Fault Tolerance for Transformer-based Language Models on ReRAM](https://mlsanity.com/paper/2401.11664/) (2024)
- [Heterogeneous Data-Centric Architectures for Modern Data-Intensive Applications: Case Studies in Machine Learning and Databases](https://mlsanity.com/paper/2205.14664/) (2022)
- [Subgraph Stationary Hardware-Software Inference Co-Design](https://mlsanity.com/paper/2306.17266/) (2023)
- [ReDy: A Novel ReRAM-centric Dynamic Quantization Approach for Energy-efficient CNN Inference](https://mlsanity.com/paper/2306.16298/) (2023)
- [DARTH-PUM: A Hybrid Processing-Using-Memory Architecture](https://mlsanity.com/paper/2602.16075/) (2026)
- [An Empirical Fault Vulnerability Exploration of ReRAM-based Process-in-Memory CNN Accelerators](https://mlsanity.com/paper/2610.07029/) (2026)

## Abstract (arXiv)

Recent works demonstrated the promise of using resistive random access memory (ReRAM) as an emerging technology to perform inherently parallel analog domain in-situ matrix-vector multiplication -- the intensive and key computation in DNNs. With weights stored in the ReRAM crossbar cells as conductance, when the input vector is applied to word lines, the matrix-vector multiplication results can be generated as the current in bit lines. A key problem is that the weight can be either positive or negative, but the in-situ computation assumes all cells on each crossbar column with the same sign. The current architectures either use two ReRAM crossbars for positive and negative weights, or add an offset to weights so that all values become positive. Neither solution is ideal: they either double the cost of crossbars, or incur extra offset circuity. To better solve this problem, this paper proposes FORMS, a fine-grained ReRAM-based DNN accelerator with polarized weights. Instead of trying to represent the positive/negative weights, our key design principle is to enforce exactly what is assumed in the in-situ computation -- ensuring that all weights in the same column of a crossbar have the same sign. It naturally avoids the cost of an additional crossbar. Such weights can be nicely generated using alternating direction method of multipliers (ADMM) regularized optimization, which can exactly enforce certain patterns in DNN weights. To achieve high accuracy, we propose to use fine-grained sub-array columns, which provide a unique opportunity for input zero-skipping, significantly avoiding unnecessary computations. It also makes the hardware much easier to implement. Putting all together, with the same optimized models, FORMS achieves significant throughput improvement and speed up in frame per second over ISAAC with similar area cost.