mlsanityData to 7 Oct 2026

GradPIM: A Practical Processing-in-DRAM Architecture for Gradient Descent

arXiv 2102.07511 ↗First posted 15 Feb 2021cs.LG cs.AR cs.DCcited by 9

Not on the map. Why?

Abstract (arXiv)

In this paper, we present GradPIM, a processing-in-memory architecture which accelerates parameter updates of deep neural networks training. As one of processing-in-memory techniques that could be realized in the near future, we propose an incremental, simple architectural design that does not invade the existing memory protocol. Extending DDR4 SDRAM to utilize bank-group parallelism makes our operation designs in processing-in-memory (PIM) module efficient in terms of hardware cost and performance. Our experimental results show that the proposed architecture can improve the performance of DNN training and greatly reduce memory bandwidth requirement while posing only a minimal amount of overhead to the protocol and DRAM area.

Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Builds on 20 of its 108 references that are in the map. First posted on arXiv 15 Feb 2021.

Builds on

20 of 108 references.

References · 20

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 4,993
  4. 2018MobileNetV2: Inverted Residuals and Linear Bottleneckscited by 4,773
  5. 2016Pyramid Scene Parsing Networkcited by 2,472
  6. 2017Mixed Precision Trainingcited by 1,135
  7. 2017In-Datacenter Performance Analysis of a Tensor Processing Unitcited by 610
  8. 2018Horovod: fast and easy distributed deep learning in TensorFlowcited by 275
  9. 2014cuDNN: Efficient Primitives for Deep Learningcited by 240
  10. 2018Training Deep Neural Networks with 8-bit Floating Point Numberscited by 163
  11. 2017SCNN: An Accelerator for Compressed-sparse Convolutional Neural Networkscited by 132
  12. 2018Highly Scalable Deep Learning Training System with Mixed-Precision: Training ImageNet in Four Minutescited by 63
8 more
  1. 2019PUMA: A Programmable Ultra-efficient Memristor-based Accelerator for Machine Learning Inferencecited by 57
  2. 2018Mixed Precision Training of Convolutional Neural Networks using Integer Operationscited by 45
  3. 2019TensorDIMM: A Practical Near-Memory Processing Architecture for Embeddings and Tensor Operations in Deep Learningcited by 42
  4. 2019Mixed Precision Training With 8-bit Floating Pointcited by 22
  5. 2018Restructuring Batch Normalization to Accelerate CNN Trainingcited by 12
  6. 2016Distributed Deep Learning Using Synchronous Stochastic Gradient Descentcited by 8
  7. 2018Mini-batch Serialization: CNN Training with Inter-layer Data Reusecited by 8
  8. 2018Massively Distributed SGD: ImageNet/ResNet-50 Training in a Flashcited by 5

Built on by

Cited by year

Citing papers per year: 2021: 1; 2022: 2; 2023: 1; 2024: 2; 2025: 3.

9 papers.

  1. 2024Smart-Infinity: Fast Large Language Model Training using Near-Storage Processing on a Real Systemcited by 14
  2. 2022Sparse Attention Acceleration with Synergistic In-Memory Pruning and On-Chip Recomputationcited by 12
  3. 2023Design Principles for Lifelong Learning AI Acceleratorscited by 9
  4. 2025Pimba: A Processing-in-Memory Acceleration for Post-Transformer Large Language Model Servingcited by 7
  5. 2024PIM-Opt: Demystifying Distributed Optimization Algorithms on a Real-World Processing-In-Memory Systemcited by 5
  6. 2025CHIME: A Case for Efficient Long-Context Attention-FC Disaggregated Inference with DIMM-PIMcited by 5
  7. 2025A Cost-Effective Near-Storage Processing Solution for Offline Inference of Long-Context LLMscited by 4
  8. 2021Demystifying BERT: Implications for Accelerator Designcited by 3
  9. 2022An Experimental Evaluation of Machine Learning Training on a Real Processing-in-Memory Systemcited by 2