# An Experimental Evaluation of Machine Learning Training on a Real Processing-in-Memory System

arXiv: https://arxiv.org/abs/2207.07886 · HTML: https://mlsanity.com/paper/2207.07886/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 44 of its 257 references that are in the map.
- First posted on arXiv 16 Jul 2022.

## Builds on (most cited first, 30 of 44)

- [A Survey of Quantization Methods for Efficient Neural Network Inference](https://mlsanity.com/paper/2103.13630/) (2021)
- [In-Datacenter Performance Analysis of a Tensor Processing Unit](https://mlsanity.com/paper/1704.04760/) (2017)
- [cuDNN: Efficient Primitives for Deep Learning](https://mlsanity.com/paper/1410.0759/) (2014)
- [PUMA: A Programmable Ultra-efficient Memristor-based Accelerator for Machine Learning Inference](https://mlsanity.com/paper/1901.10351/) (2019)
- [RecNMP: Accelerating Personalized Recommendation with Near-Memory Processing](https://mlsanity.com/paper/1912.12953/) (2019)
- [TensorDIMM: A Practical Near-Memory Processing Architecture for Embeddings and Tensor Operations in Deep Learning](https://mlsanity.com/paper/1908.03072/) (2019)
- [A Survey on Large-scale Machine Learning](https://mlsanity.com/paper/2008.03911/) (2020)
- [Neural Cache: Bit-Serial In-Cache Acceleration of Deep Neural Networks](https://mlsanity.com/paper/1805.03718/) (2018)
- [A Modern Primer on Processing in Memory](https://mlsanity.com/paper/2012.03112/) (2020)
- [Google Neural Network Models for Edge Devices: Analyzing and Mitigating Machine Learning Inference Bottlenecks](https://mlsanity.com/paper/2109.14320/) (2021)
- [Processing Data Where It Makes Sense: Enabling In-Memory Computation](https://mlsanity.com/paper/1903.03988/) (2019)
- [PANTHER: A Programmable Architecture for Neural Network Training Harnessing Energy-efficient ReRAM](https://mlsanity.com/paper/1912.11516/) (2019)
- [SISA: Set-Centric Instruction Set Architecture for Graph Mining on Processing-in-Memory Systems](https://mlsanity.com/paper/2104.07582/) (2021)
- [Multiscale Co-Design Analysis of Energy, Latency, Area, and Accuracy of a ReRAM Analog Neural Training Accelerator](https://mlsanity.com/paper/1707.09952/) (2017)
- [Benchmarking Memory-Centric Computing Systems: Analysis of Real Processing-in-Memory Hardware](https://mlsanity.com/paper/2110.01709/) (2021)
- [GRIM-Filter: Fast Seed Location Filtering in DNA Read Mapping Using Processing-in-Memory Technologies](https://mlsanity.com/paper/1711.01177/) (2017)
- [NATSA: A Near-Data Processing Accelerator for Time Series Analysis](https://mlsanity.com/paper/2010.02079/) (2020)
- [Fast Stencil-Code Computation on a Wafer-Scale Processor](https://mlsanity.com/paper/2010.03660/) (2020)
- [GradPIM: A Practical Processing-in-DRAM Architecture for Gradient Descent](https://mlsanity.com/paper/2102.07511/) (2021)
- [DAMOV: A New Methodology and Benchmark Suite for Evaluating Data Movement Bottlenecks](https://mlsanity.com/paper/2105.03725/) (2021)
- [GenASM: A High-Performance, Low-Power Approximate String Matching Acceleration Framework for Genome Sequence Analysis](https://mlsanity.com/paper/2009.07692/) (2020)
- [SynCron: Efficient Synchronization Support for Near-Data-Processing Architectures](https://mlsanity.com/paper/2101.07557/) (2021)
- [Benchmarking a New Paradigm: An Experimental Analysis of a Real Processing-in-Memory Architecture](https://mlsanity.com/paper/2105.03814/) (2021)
- [NERO: A Near High-Bandwidth Memory Stencil Accelerator for Weather Prediction Modeling](https://mlsanity.com/paper/2009.08241/) (2020)
- [QUAC-TRNG: High-Throughput True Random Number Generation Using Quadruple Row Activation in Commodity DRAM Chips](https://mlsanity.com/paper/2105.08955/) (2021)
- [FPGA-Based Near-Memory Acceleration of Modern Data-Intensive Applications](https://mlsanity.com/paper/2106.06433/) (2021)
- [Sibyl: Adaptive and Extensible Data Placement in Hybrid Storage Systems Using Online Reinforcement Learning](https://mlsanity.com/paper/2205.07394/) (2022)
- [One-step regression and classification with crosspoint resistive memory arrays](https://mlsanity.com/paper/2005.01988/) (2020)
- [Buddy-RAM: Improving the Performance and Efficiency of Bulk Bitwise Operations Using DRAM](https://mlsanity.com/paper/1611.09988/) (2016)
- [Neurostream: Scalable and Energy Efficient Deep Learning with Smart Memory Cubes](https://mlsanity.com/paper/1701.06420/) (2017)

## Built on by (most cited first, 2 of 2)

- [PIM-Opt: Demystifying Distributed Optimization Algorithms on a Real-World Processing-In-Memory System](https://mlsanity.com/paper/2404.07164/) (2024)
- [KD-PINN: Knowledge-Distilled PINNs for ultra-low-latency real-time neural PDE solvers](https://mlsanity.com/paper/2512.13336/) (2025)

## Abstract (arXiv)

Training machine learning (ML) algorithms is a computationally intensive process, which is frequently memory-bound due to repeatedly accessing large training datasets. As a result, processor-centric systems (e.g., CPU, GPU) suffer from costly data movement between memory units and processing units, which consumes large amounts of energy and execution cycles. Memory-centric computing systems, i.e., with processing-in-memory (PIM) capabilities, can alleviate this data movement bottleneck. Our goal is to understand the potential of modern general-purpose PIM architectures to accelerate ML training. To do so, we (1) implement several representative classic ML algorithms (namely, linear regression, logistic regression, decision tree, K-Means clustering) on a real-world general-purpose PIM architecture, (2) rigorously evaluate and characterize them in terms of accuracy, performance and scaling, and (3) compare to their counterpart implementations on CPU and GPU. Our evaluation on a real memory-centric computing system with more than 2500 PIM cores shows that general-purpose PIM architectures can greatly accelerate memory-bound ML workloads, when the necessary operations and datatypes are natively supported by PIM hardware. For example, our PIM implementation of decision tree is 27× faster than a state-of-the-art CPU version on an 8-core Intel Xeon, and 1.34× faster than a state-of-the-art GPU version on an NVIDIA A100. Our K-Means clustering on PIM is 2.8× and 3.2× than state-of-the-art CPU and GPU versions, respectively. To our knowledge, our work is the first one to evaluate ML training on a real-world PIM architecture. We conclude with key observations, takeaways, and recommendations that can inspire users of ML workloads, programmers of PIM architectures, and hardware designers & architects of future memory-centric computing systems.