# Measurement-driven neural-network training for integrated magnetic tunnel junction arrays

arXiv: https://arxiv.org/abs/2312.06446 · HTML: https://mlsanity.com/paper/2312.06446/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 19 of its 83 references that are in the map.
- First posted on arXiv 11 Dec 2023.

## Builds on (most cited first, 19 of 19)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding](https://mlsanity.com/paper/1510.00149/) (2015)
- [Visualizing the Loss Landscape of Neural Nets](https://mlsanity.com/paper/1712.09913/) (2017)
- [Quantized Neural Networks: Training Neural Networks with Low Precision Weights and Activations](https://mlsanity.com/paper/1609.07061/) (2016)
- [Binary Neural Networks: A Survey](https://mlsanity.com/paper/2004.03333/) (2020)
- [Deep physical neural networks enabled by a backpropagation algorithm for arbitrary physical systems](https://mlsanity.com/paper/2104.13386/) (2021)
- [Benchmark Analysis of Representative Deep Neural Network Architectures](https://mlsanity.com/paper/1810.00736/) (2018)
- [Sparse learning of stochastic dynamic equations](https://mlsanity.com/paper/1712.02432/) (2017)
- [Brain-inspired computing: We need a master plan](https://mlsanity.com/paper/2104.14517/) (2021)
- [Hardware-aware training for large-scale and diverse deep learning inference workloads using in-memory computing-based accelerators](https://mlsanity.com/paper/2302.08469/) (2023)
- [Hardware and Software Optimizations for Accelerating Deep Neural Networks: Survey of Current Trends, Challenges, and the Road Ahead](https://mlsanity.com/paper/2012.11233/) (2020)
- [Acceleration of Deep Neural Network Training with Resistive Cross-Point Devices](https://mlsanity.com/paper/1603.07341/) (2016)
- [Streaming Batch Eigenupdates for Hardware Neuromorphic Networks](https://mlsanity.com/paper/1903.01635/) (2019)
- [Implementing Binarized Neural Networks with Magnetoresistive RAM without Error Correction](https://mlsanity.com/paper/1908.04085/) (2019)
- [Fast Implementation of 4-bit Convolutional Neural Networks for Mobile Devices](https://mlsanity.com/paper/2009.06488/) (2020)
- [Neural-like computing with populations of superparamagnetic basis functions](https://mlsanity.com/paper/1610.09394/) (2016)
- [Bit Error Tolerance Metrics for Binarized Neural Networks](https://mlsanity.com/paper/2102.01344/) (2021)
- [Experimental Demonstration of Neuromorphic Network with STT MTJ Synapses](https://mlsanity.com/paper/2112.04749/) (2021)
- [Implementation of a Binary Neural Network on a Passive Array of Magnetic Tunnel Junctions](https://mlsanity.com/paper/2112.09159/) (2021)

## Built on by (most cited first, 3 of 3)

- [XBTorch: A Unified Framework for Modeling and Co-Design of Crossbar-Based Deep Learning Accelerators](https://mlsanity.com/paper/2601.07086/) (2026)
- [Layer Ensemble Averaging for Improving Memristor-Based Artificial Neural Network Performance](https://mlsanity.com/paper/2404.15621/) (2024)
- [Training a multilayer dynamical spintronic network with standard machine learning tools to perform time series classification](https://mlsanity.com/paper/2408.02835/) (2024)

## Abstract (arXiv)

The increasing scale of neural networks needed to support more complex applications has led to an increasing requirement for area- and energy-efficient hardware. One route to meeting the budget for these applications is to circumvent the von Neumann bottleneck by performing computation in or near memory. An inevitability of transferring neural networks onto hardware is that non-idealities such as device-to-device variations or poor device yield impact performance. Methods such as hardware-aware training, where substrate non-idealities are incorporated during network training, are one way to recover performance at the cost of solution generality. In this work, we demonstrate inference on hardware neural networks consisting of 20,000 magnetic tunnel junction arrays integrated on a complementary metal-oxide-semiconductor chips that closely resembles market-ready spin transfer-torque magnetoresistive random access memory technology. Using 36 dies, each containing a crossbar array with its own non-idealities, we show that even a small number of defects in physically mapped networks significantly degrades the performance of networks trained without defects and show that, at the cost of generality, hardware-aware training accounting for specific defects on each die can recover to comparable performance with ideal networks. We then demonstrate a robust training method that extends hardware-aware training to statistics-aware training, producing network weights that perform well on most defective dies regardless of their specific defect locations. When evaluated on the 36 physical dies, statistics-aware trained solutions can achieve a mean misclassification error on the MNIST dataset that differs from the software-baseline by only 2 %. This statistics-aware training method could be generalized to networks with many layers that are mapped to hardware suited for industry-ready applications.