# Acceleration of Deep Neural Network Training with Resistive Cross-Point Devices

arXiv: https://arxiv.org/abs/1603.07341 · HTML: https://mlsanity.com/paper/1603.07341/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Mar 2016.

## Built on by (most cited first, 29 of 29)

- [Brain-inspired computing: We need a master plan](https://mlsanity.com/paper/2104.14517/) (2021)
- [Hardware-aware training for large-scale and diverse deep learning inference workloads using in-memory computing-based accelerators](https://mlsanity.com/paper/2302.08469/) (2023)
- [A flexible and fast PyTorch toolkit for simulating training and inference on analog crossbar arrays](https://mlsanity.com/paper/2104.02184/) (2021)
- [Fast offset corrected in-memory training](https://mlsanity.com/paper/2303.04721/) (2023)
- [Using the IBM Analog In-Memory Hardware Acceleration Kit for Neural Network Training and Inference](https://mlsanity.com/paper/2307.09357/) (2023)
- [Resource-Efficient Deep Learning: A Survey on Model-, Arithmetic-, and Implementation-Level Techniques](https://mlsanity.com/paper/2112.15131/) (2021)
- [Towards Exact Gradient-based Training on Analog In-memory Computing](https://mlsanity.com/paper/2406.12774/) (2024)
- [Analog In-memory Training on General Non-ideal Resistive Elements: The Impact of Response Functions](https://mlsanity.com/paper/2502.06309/) (2025)
- [BPLight-CNN: A Photonics-based Backpropagation Accelerator for Deep Learning](https://mlsanity.com/paper/2102.10140/) (2021)
- [Neural Network Training with Asymmetric Crosspoint Elements](https://mlsanity.com/paper/2201.13377/) (2022)
- [Measurement-driven neural-network training for integrated magnetic tunnel junction arrays](https://mlsanity.com/paper/2312.06446/) (2023)
- [In-memory Training on Analog Devices with Limited Conductance States via Multi-tile Residual Learning](https://mlsanity.com/paper/2510.02516/) (2025)
- [Variance-Aware Noisy Training: Hardening DNNs against Unstable Analog Computations](https://mlsanity.com/paper/2503.16183/) (2025)
- [Dynamic Symmetric Point Tracking: Tackling Non-ideal Reference in Analog In-memory Training](https://mlsanity.com/paper/2602.21321/) (2026)
- [Hybrid In-memory Computing Architecture for the Training of Deep Neural Networks](https://mlsanity.com/paper/2102.05271/) (2021)
- [Variation Aware Training of Hybrid Precision Neural Networks with 28nm HKMG FeFET Based Synaptic Core](https://mlsanity.com/paper/2202.10912/) (2022)
- [Device Modeling Bias in ReRAM-based Neural Network Simulations](https://mlsanity.com/paper/2211.15925/) (2022)
- [Assessing the Performance of Analog Training for Transfer Learning](https://mlsanity.com/paper/2505.11067/) (2025)
- [Energy-convergence trade off for the training of neural networks on bio-inspired hardware](https://mlsanity.com/paper/2509.18121/) (2025)
- [Making Analog Training Scale: Co-Designing Mapping, Optimizer, and Converters](https://mlsanity.com/paper/2609.36584/) (2026)
- [LiteCON: An All-Photonic Neuromorphic Accelerator for Energy-efficient Deep Learning (Preprint)](https://mlsanity.com/paper/2206.13861/) (2022)
- [A temporally and spatially local spike-based backpropagation algorithm to enable training in hardware](https://mlsanity.com/paper/2207.09755/) (2022)
- [Binary stochasticity enabled highly efficient neuromorphic deep learning achieves better-than-software accuracy](https://mlsanity.com/paper/2304.12866/) (2023)
- [System-level Impact of Non-Ideal Program-Time of Charge Trap Flash (CTF) on Deep Neural Network](https://mlsanity.com/paper/2402.09792/) (2024)
- [On the Convergence Theory of Pipeline Gradient-based Analog In-memory Training](https://mlsanity.com/paper/2410.15155/) (2024)
- [Personalized Spiking Neural Networks with Ferroelectric Synapses for EEG Signal Processing](https://mlsanity.com/paper/2601.00020/) (2025)
- [Online local learning for generative thermodynamic computing](https://mlsanity.com/paper/2609.15439/) (2026)
- [Physical Muon: Orthogonalization as an Equilibrium Computation](https://mlsanity.com/paper/2609.37525/) (2026)
- [Robust Parameter-Efficient LLM Adaptation on Analog Hardware](https://mlsanity.com/paper/2610.05318/) (2026)

## Abstract (arXiv)

In recent years, deep neural networks (DNN) have demonstrated significant business impact in large scale analysis and classification tasks such as speech recognition, visual object detection, pattern extraction, etc. Training of large DNNs, however, is universally considered as time consuming and computationally intensive task that demands datacenter-scale computational resources recruited for many days. Here we propose a concept of resistive processing unit (RPU) devices that can potentially accelerate DNN training by orders of magnitude while using much less power. The proposed RPU device can store and update the weight values locally thus minimizing data movement during training and allowing to fully exploit the locality and the parallelism of the training algorithm. We identify the RPU device and system specifications for implementation of an accelerator chip for DNN training in a realistic CMOS-compatible technology. For large DNNs with about 1 billion weights this massively parallel RPU architecture can achieve acceleration factors of 30,000X compared to state-of-the-art microprocessors while providing power efficiency of 84,000 GigaOps/s/W. Problems that currently require days of training on a datacenter-size cluster with thousands of machines can be addressed within hours on a single RPU accelerator. A system consisted of a cluster of RPU accelerators will be able to tackle Big Data problems with trillions of parameters that is impossible to address today like, for example, natural speech recognition and translation between all world languages, real-time analytics on large streams of business and scientific data, integration and analysis of multimodal sensory data flows from massive number of IoT (Internet of Things) sensors.