# Neural Cache: Bit-Serial In-Cache Acceleration of Deep Neural Networks

arXiv: https://arxiv.org/abs/1805.03718 · HTML: https://mlsanity.com/paper/1805.03718/

## Key facts

- Cited by 32 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 9 May 2018.

## Built on by (most cited first, 30 of 32)

- [LLM in a flash: Efficient Large Language Model Inference with Limited Memory](https://mlsanity.com/paper/2312.11514/) (2023)
- [NIPQ: Noise proxy-based Integrated Pseudo-Quantization](https://mlsanity.com/paper/2206.00820/) (2022)
- [Tender: Accelerating Large Language Models via Tensor Decomposition and Runtime Requantization](https://mlsanity.com/paper/2406.12930/) (2024)
- [SmartSAGE: Training Large-scale Graph Neural Networks using In-Storage Processing Architectures](https://mlsanity.com/paper/2205.04711/) (2022)
- [Reconsidering the Energy Efficiency of Spiking Neural Networks Inference from Analytical Perspectives](https://mlsanity.com/paper/2409.08290/) (2024)
- [Sparse Attention Acceleration with Synergistic In-Memory Pruning and On-Chip Recomputation](https://mlsanity.com/paper/2209.00606/) (2022)
- [Accelerating Attention through Gradient-Based Learned Runtime Pruning](https://mlsanity.com/paper/2204.03227/) (2022)
- [Accelerating Neural Network Inference with Processing-in-DRAM: From the Edge to the Cloud](https://mlsanity.com/paper/2209.08938/) (2022)
- [FlexiBit: Fully Flexible Precision Bit-parallel Accelerator Architecture for Arbitrary Mixed Precision AI](https://mlsanity.com/paper/2411.18065/) (2024)
- [Rethinking Floating Point Overheads for Mixed Precision DNN Accelerators](https://mlsanity.com/paper/2101.11748/) (2021)
- [A Survey of Near-Data Processing Architectures for Neural Networks](https://mlsanity.com/paper/2112.12630/) (2021)
- [Machine Learning Training on a Real Processing-in-Memory System](https://mlsanity.com/paper/2206.06022/) (2022)
- [CIM-MLC: A Multi-level Compilation Stack for Computing-In-Memory Accelerators](https://mlsanity.com/paper/2401.12428/) (2024)
- [An Experimental Evaluation of Machine Learning Training on a Real Processing-in-Memory System](https://mlsanity.com/paper/2207.07886/) (2022)
- [OISMA: On-the-fly In-memory Stochastic Multiplication Architecture for Matrix-Multiplication Workloads](https://mlsanity.com/paper/2508.08822/) (2025)
- [AQPIM: Breaking the PIM Capacity Wall for LLMs with In-Memory Activation Quantization](https://mlsanity.com/paper/2604.18137/) (2026)
- [ODIN: A Bit-Parallel Stochastic Arithmetic Based Accelerator for In-Situ Neural Network Processing in Phase Change RAM](https://mlsanity.com/paper/2103.03953/) (2021)
- [ATRIA: A Bit-Parallel Stochastic Arithmetic Based Accelerator for In-DRAM CNN Processing](https://mlsanity.com/paper/2105.12781/) (2021)
- [Heterogeneous Data-Centric Architectures for Modern Data-Intensive Applications: Case Studies in Machine Learning and Databases](https://mlsanity.com/paper/2205.14664/) (2022)
- [RedBit: An End-to-End Flexible Framework for Evaluating the Accuracy of Quantized CNNs](https://mlsanity.com/paper/2301.06193/) (2023)
- [TransPimLib: A Library for Efficient Transcendental Functions on Processing-in-Memory Systems](https://mlsanity.com/paper/2304.01951/) (2023)
- [DAISM: Digital Approximate In-SRAM Multiplier-based Accelerator for DNN Training and Inference](https://mlsanity.com/paper/2305.07376/) (2023)
- [ARTEMIS: A Mixed Analog-Stochastic In-DRAM Accelerator for Transformer Neural Networks](https://mlsanity.com/paper/2407.12638/) (2024)
- [An Event-Based Digital Compute-In-Memory Accelerator with Flexible Operand Resolution and Layer-Wise Weight/Output Stationarity](https://mlsanity.com/paper/2410.23082/) (2024)
- [SAIL: SRAM-Accelerated LLM Inference System with Lookup-Table-based GEMV](https://mlsanity.com/paper/2509.25853/) (2025)
- [DISCA: A Digital In-memory Stochastic Computing Architecture Using A Compressed Bent-Pyramid Format](https://mlsanity.com/paper/2511.17265/) (2025)
- [OPTIMA: Design-Space Exploration of Discharge-Based In-SRAM Computing: Quantifying Energy-Accuracy Trade-Offs](https://mlsanity.com/paper/2411.06846/) (2024)
- [PREBA: A Hardware/Software Co-Design for Multi-Instance GPU based AI Inference Servers](https://mlsanity.com/paper/2411.19114/) (2024)
- [Hardware-software co-exploration with racetrack memory based in-memory computing for CNN inference in embedded systems](https://mlsanity.com/paper/2507.01429/) (2025)
- [Time--to--Digital Converter (TDC)--Based Resonant Compute--in--Memory for INT8 CNNs with Layer--Optimized SRAM Mapping](https://mlsanity.com/paper/2601.00434/) (2026)

## Abstract (arXiv)

This paper presents the Neural Cache architecture, which re-purposes cache structures to transform them into massively parallel compute units capable of running inferences for Deep Neural Networks. Techniques to do in-situ arithmetic in SRAM arrays, create efficient data mapping and reducing data movement are proposed. The Neural Cache architecture is capable of fully executing convolutional, fully connected, and pooling layers in-cache. The proposed architecture also supports quantization in-cache. Our experimental results show that the proposed architecture can improve inference latency by 18.3x over state-of-art multi-core CPU (Xeon E5), 7.7x over server class GPU (Titan Xp), for Inception v3 model. Neural Cache improves inference throughput by 12.4x over CPU (2.2x over GPU), while reducing power consumption by 50% over CPU (53% over GPU).