# Training Deep Convolutional Neural Networks with Resistive Cross-Point Devices

arXiv: https://arxiv.org/abs/1705.08014 · HTML: https://mlsanity.com/paper/1705.08014/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 22 May 2017.

## Built on by (most cited first, 18 of 18)

- [Hardware-aware training for large-scale and diverse deep learning inference workloads using in-memory computing-based accelerators](https://mlsanity.com/paper/2302.08469/) (2023)
- [Fast offset corrected in-memory training](https://mlsanity.com/paper/2303.04721/) (2023)
- [Using the IBM Analog In-Memory Hardware Acceleration Kit for Neural Network Training and Inference](https://mlsanity.com/paper/2307.09357/) (2023)
- [Resource-Efficient Deep Learning: A Survey on Model-, Arithmetic-, and Implementation-Level Techniques](https://mlsanity.com/paper/2112.15131/) (2021)
- [Spatial-SpinDrop: Spatial Dropout-based Binary Bayesian Neural Network with Spintronics Implementation](https://mlsanity.com/paper/2306.10185/) (2023)
- [Towards Exact Gradient-based Training on Analog In-memory Computing](https://mlsanity.com/paper/2406.12774/) (2024)
- [Scale-Dropout: Estimating Uncertainty in Deep Neural Networks Using Stochastic Scale](https://mlsanity.com/paper/2311.15816/) (2023)
- [Analog In-memory Training on General Non-ideal Resistive Elements: The Impact of Response Functions](https://mlsanity.com/paper/2502.06309/) (2025)
- [BPLight-CNN: A Photonics-based Backpropagation Accelerator for Deep Learning](https://mlsanity.com/paper/2102.10140/) (2021)
- [Neural Network Training with Asymmetric Crosspoint Elements](https://mlsanity.com/paper/2201.13377/) (2022)
- [Hybrid In-memory Computing Architecture for the Training of Deep Neural Networks](https://mlsanity.com/paper/2102.05271/) (2021)
- [Dither computing: a hybrid deterministic-stochastic computing framework](https://mlsanity.com/paper/2102.10732/) (2021)
- [EcoFlow: Efficient Convolutional Dataflows for Low-Power Neural Network Accelerators](https://mlsanity.com/paper/2202.02310/) (2022)
- [Making Analog Training Scale: Co-Designing Mapping, Optimizer, and Converters](https://mlsanity.com/paper/2609.36584/) (2026)
- [ReDy: A Novel ReRAM-centric Dynamic Quantization Approach for Energy-efficient CNN Inference](https://mlsanity.com/paper/2306.16298/) (2023)
- [Quantization of Deep Neural Networks to facilitate self-correction of weights on Phase Change Memory-based analog hardware](https://mlsanity.com/paper/2310.00337/) (2023)
- [NeuSpin: Design of a Reliable Edge Neuromorphic System Based on Spintronics for Green AI](https://mlsanity.com/paper/2401.06195/) (2024)
- [On the Convergence Theory of Pipeline Gradient-based Analog In-memory Training](https://mlsanity.com/paper/2410.15155/) (2024)

## Abstract (arXiv)

In a previous work we have detailed the requirements to obtain a maximal performance benefit by implementing fully connected deep neural networks (DNN) in form of arrays of resistive devices for deep learning. This concept of Resistive Processing Unit (RPU) devices we extend here towards convolutional neural networks (CNNs). We show how to map the convolutional layers to RPU arrays such that the parallelism of the hardware can be fully utilized in all three cycles of the backpropagation algorithm. We find that the noise and bound limitations imposed due to analog nature of the computations performed on the arrays effect the training accuracy of the CNNs. Noise and bound management techniques are presented that mitigate these problems without introducing any additional complexity in the analog circuits and can be addressed by the digital circuits. In addition, we discuss digitally programmable update management and device variability reduction techniques that can be used selectively for some of the layers in a CNN. We show that combination of all those techniques enables a successful application of the RPU concept for training CNNs. The techniques discussed here are more general and can be applied beyond CNN architectures and therefore enables applicability of RPU approach for large class of neural network architectures.