# PUMA: A Programmable Ultra-efficient Memristor-based Accelerator for Machine Learning Inference

arXiv: https://arxiv.org/abs/1901.10351 · HTML: https://mlsanity.com/paper/1901.10351/

## Key facts

- Cited by 61 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 29 Jan 2019.

## Built on by (most cited first, 30 of 61)

- [X-Former: In-Memory Acceleration of Transformers](https://mlsanity.com/paper/2303.07470/) (2023)
- [Reducing Energy Bloat in Large Model Training](https://mlsanity.com/paper/2312.06902/) (2023)
- [Memory Is All You Need: An Overview of Compute-in-Memory Architectures for Accelerating Large Language Model Inference](https://mlsanity.com/paper/2406.08413/) (2024)
- [Automated Deep Learning: Neural Architecture Search Is Not the End](https://mlsanity.com/paper/2112.09245/) (2021)
- [FORMS: Fine-grained Polarized ReRAM-based In-situ Computation for Mixed-signal DNN Accelerator](https://mlsanity.com/paper/2106.09144/) (2021)
- [Using the IBM Analog In-Memory Hardware Acceleration Kit for Neural Network Training and Inference](https://mlsanity.com/paper/2307.09357/) (2023)
- [GradPIM: A Practical Processing-in-DRAM Architecture for Gradient Descent](https://mlsanity.com/paper/2102.07511/) (2021)
- [On the Accuracy of Analog Neural Network Inference Accelerators](https://mlsanity.com/paper/2109.01262/) (2021)
- [SpikeSim: An end-to-end Compute-in-Memory Hardware Evaluation Tool for Benchmarking Spiking Neural Networks](https://mlsanity.com/paper/2210.12899/) (2022)
- [AnalogNets: ML-HW Co-Design of Noise-robust TinyML Models and Always-On Analog Compute-in-Memory Accelerator](https://mlsanity.com/paper/2111.06503/) (2021)
- [Interconnect Parasitics and Partitioning in Fully-Analog In-Memory Computing Architectures](https://mlsanity.com/paper/2201.12480/) (2022)
- [Bring Your Own Codegen to Deep Learning Compiler](https://mlsanity.com/paper/2105.03215/) (2021)
- [Towards Efficient In-memory Computing Hardware for Quantized Neural Networks: State-of-the-art, Open Challenges and Perspectives](https://mlsanity.com/paper/2307.03936/) (2023)
- [MRAM-based Analog Sigmoid Function for In-memory Computing](https://mlsanity.com/paper/2204.09918/) (2022)
- [Heterogeneous Integration of In-Memory Analog Computing Architectures with Tensor Processing Units](https://mlsanity.com/paper/2304.09258/) (2023)
- [Efficiency-driven Hardware Optimization for Adversarially Robust Neural Networks](https://mlsanity.com/paper/2105.04003/) (2021)
- [NAX: Co-Designing Neural Network and Hardware Architecture for Memristive Xbar based Computing Systems](https://mlsanity.com/paper/2106.12125/) (2021)
- [CIM-MLC: A Multi-level Compilation Stack for Computing-In-Memory Accelerators](https://mlsanity.com/paper/2401.12428/) (2024)
- [Reducing the Barriers to Entry for Foundation Model Training](https://mlsanity.com/paper/2404.08811/) (2024)
- [BF-IMNA: A Bit Fluid In-Memory Neural Architecture for Neural Network Acceleration](https://mlsanity.com/paper/2411.01417/) (2024)
- [An Experimental Evaluation of Machine Learning Training on a Real Processing-in-Memory System](https://mlsanity.com/paper/2207.07886/) (2022)
- [HCiM: ADC-Less Hybrid Analog-Digital Compute in Memory Accelerator for Deep Learning Workloads](https://mlsanity.com/paper/2403.13577/) (2024)
- [StoX-Net: Stochastic Processing of Partial Sums for Efficient In-Memory Computing DNN Accelerators](https://mlsanity.com/paper/2407.12378/) (2024)
- [COMPASS: A Compiler Framework for Resource-Constrained Crossbar-Array Based In-Memory Deep Learning Accelerators](https://mlsanity.com/paper/2501.06780/) (2025)
- [MINIMALIST: switched-capacitor circuits for efficient in-memory computation of gated recurrent units](https://mlsanity.com/paper/2505.08599/) (2025)
- [EcoFlow: Efficient Convolutional Dataflows for Low-Power Neural Network Accelerators](https://mlsanity.com/paper/2202.02310/) (2022)
- [Thermal Heating in ReRAM Crossbar Arrays: Challenges and Solutions](https://mlsanity.com/paper/2212.13707/) (2022)
- [TransPimLib: A Library for Efficient Transcendental Functions on Processing-in-Memory Systems](https://mlsanity.com/paper/2304.01951/) (2023)
- [IMAC-Sim: A Circuit-level Simulator For In-Memory Analog Computing Architectures](https://mlsanity.com/paper/2304.09252/) (2023)
- [Comprehensive Benchmarking of Binary Neural Networks on NVM Crossbar Architectures](https://mlsanity.com/paper/2308.06227/) (2023)

## Abstract (arXiv)

Memristor crossbars are circuits capable of performing analog matrix-vector multiplications, overcoming the fundamental energy efficiency limitations of digital logic. They have been shown to be effective in special-purpose accelerators for a limited set of neural network applications. We present the Programmable Ultra-efficient Memristor-based Accelerator (PUMA) which enhances memristor crossbars with general purpose execution units to enable the acceleration of a wide variety of Machine Learning (ML) inference workloads. PUMA's microarchitecture techniques exposed through a specialized Instruction Set Architecture (ISA) retain the efficiency of in-memory computing and analog circuitry, without compromising programmability. We also present the PUMA compiler which translates high-level code to PUMA ISA. The compiler partitions the computational graph and optimizes instruction scheduling and register allocation to generate code for large and complex workloads to run on thousands of spatial cores. We have developed a detailed architecture simulator that incorporates the functionality, timing, and power models of PUMA's components to evaluate performance and energy consumption. A PUMA accelerator running at 1 GHz can reach area and power efficiency of 577~GOPS/s/mm2 and 837~GOPS/s/W, respectively. Our evaluation of diverse ML applications from image recognition, machine translation, and language modelling (5M-800M synapses) shows that PUMA achieves up to 2,446× energy and 66× latency improvement for inference compared to state-of-the-art GPUs. Compared to an application-specific memristor-based accelerator, PUMA incurs small energy overheads at similar inference latency and added programmability.