# Algorithm for Training Neural Networks on Resistive Device Arrays

arXiv: https://arxiv.org/abs/1909.07908 · HTML: https://mlsanity.com/paper/1909.07908/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 17 Sep 2019.

## Built on by (most cited first, 19 of 19)

- [A flexible and fast PyTorch toolkit for simulating training and inference on analog crossbar arrays](https://mlsanity.com/paper/2104.02184/) (2021)
- [Fast offset corrected in-memory training](https://mlsanity.com/paper/2303.04721/) (2023)
- [Using the IBM Analog In-Memory Hardware Acceleration Kit for Neural Network Training and Inference](https://mlsanity.com/paper/2307.09357/) (2023)
- [Towards Exact Gradient-based Training on Analog In-memory Computing](https://mlsanity.com/paper/2406.12774/) (2024)
- [Analog In-memory Training on General Non-ideal Resistive Elements: The Impact of Response Functions](https://mlsanity.com/paper/2502.06309/) (2025)
- [Neural Network Training with Asymmetric Crosspoint Elements](https://mlsanity.com/paper/2201.13377/) (2022)
- [In-memory Training on Analog Devices with Limited Conductance States via Multi-tile Residual Learning](https://mlsanity.com/paper/2510.02516/) (2025)
- [Dynamic Symmetric Point Tracking: Tackling Non-ideal Reference in Analog In-memory Training](https://mlsanity.com/paper/2602.21321/) (2026)
- [Assessing the Performance of Analog Training for Transfer Learning](https://mlsanity.com/paper/2505.11067/) (2025)
- [Energy-convergence trade off for the training of neural networks on bio-inspired hardware](https://mlsanity.com/paper/2509.18121/) (2025)
- [XBTorch: A Unified Framework for Modeling and Co-Design of Crossbar-Based Deep Learning Accelerators](https://mlsanity.com/paper/2601.07086/) (2026)
- [Making Analog Training Scale: Co-Designing Mapping, Optimizer, and Converters](https://mlsanity.com/paper/2609.36584/) (2026)
- [Binary stochasticity enabled highly efficient neuromorphic deep learning achieves better-than-software accuracy](https://mlsanity.com/paper/2304.12866/) (2023)
- [On the Convergence Theory of Pipeline Gradient-based Analog In-memory Training](https://mlsanity.com/paper/2410.15155/) (2024)
- [Bruno: Backpropagation Running Undersampled for Novel device Optimization](https://mlsanity.com/paper/2505.17791/) (2025)
- [Analog Physical Systems Can Exhibit Double Descent](https://mlsanity.com/paper/2511.17825/) (2025)
- [Online local learning for generative thermodynamic computing](https://mlsanity.com/paper/2609.15439/) (2026)
- [Physical Muon: Orthogonalization as an Equilibrium Computation](https://mlsanity.com/paper/2609.37525/) (2026)
- [Robust Parameter-Efficient LLM Adaptation on Analog Hardware](https://mlsanity.com/paper/2610.05318/) (2026)

## Abstract (arXiv)

Hardware architectures composed of resistive cross-point device arrays can provide significant power and speed benefits for deep neural network training workloads using stochastic gradient descent (SGD) and backpropagation (BP) algorithm. The training accuracy on this imminent analog hardware however strongly depends on the switching characteristics of the cross-point elements. One of the key requirements is that these resistive devices must change conductance in a symmetrical fashion when subjected to positive or negative pulse stimuli. Here, we present a new training algorithm, so-called the "Tiki-Taka" algorithm, that eliminates this stringent symmetry requirement. We show that device asymmetry introduces an unintentional implicit cost term into the SGD algorithm, whereas in the "Tiki-Taka" algorithm a coupled dynamical system simultaneously minimizes the original objective function of the neural network and the unintentional cost term due to device asymmetry in a self-consistent fashion. We tested the validity of this new algorithm on a range of network architectures such as fully connected, convolutional and LSTM networks. Simulation results on these various networks show that whatever accuracy is achieved using the conventional SGD algorithm with symmetric (ideal) device switching characteristics the same accuracy is also achieved using the "Tiki-Taka" algorithm with non-symmetric (non-ideal) device switching characteristics. Moreover, all the operations performed on the arrays are still parallel and therefore the implementation cost of this new algorithm on array architectures is minimal; and it maintains the aforementioned power and speed benefits. These algorithmic improvements are crucial to relax the material specification and to realize technologically viable resistive crossbar arrays that outperform digital accelerators for similar training tasks.