# Layer Ensemble Averaging for Improving Memristor-Based Artificial Neural Network Performance

arXiv: https://arxiv.org/abs/2404.15621 · HTML: https://mlsanity.com/paper/2404.15621/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 11 of its 42 references that are in the map.
- First posted on arXiv 24 Apr 2024.

## Builds on (most cited first, 11 of 11)

- [Overcoming catastrophic forgetting in neural networks](https://mlsanity.com/paper/1612.00796/) (2016)
- [Optuna: A Next-generation Hyperparameter Optimization Framework](https://mlsanity.com/paper/1907.10902/) (2019)
- [An Overview of Multi-Task Learning in Deep Neural Networks](https://mlsanity.com/paper/1706.05098/) (2017)
- [BRECQ: Pushing the Limit of Post-Training Quantization by Block Reconstruction](https://mlsanity.com/paper/2102.05426/) (2021)
- [The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bits](https://mlsanity.com/paper/2402.17764/) (2024)
- [Ternary Weight Networks](https://mlsanity.com/paper/1605.04711/) (2016)
- [BitNet: Scaling 1-bit Transformers for Large Language Models](https://mlsanity.com/paper/2310.11453/) (2023)
- [The Yin-Yang dataset](https://mlsanity.com/paper/2102.08211/) (2021)
- [In-memory Realization of In-situ Few-shot Continual Learning with a Dynamically Evolving Explicit Memory](https://mlsanity.com/paper/2207.06810/) (2022)
- [Measurement-driven neural-network training for integrated magnetic tunnel junction arrays](https://mlsanity.com/paper/2312.06446/) (2023)
- [Methodology for Realizing VMM with Binary RRAM Arrays: Experimental Demonstration of Binarized-ADALINE Using OxRAM Crossbar](https://mlsanity.com/paper/2006.05657/) (2020)

## Abstract (arXiv)

Artificial neural networks have advanced due to scaling dimensions, but conventional computing faces inefficiency due to the von Neumann bottleneck. In-memory computation architectures, like memristors, offer promise but face challenges due to hardware non-idealities. This work proposes and experimentally demonstrates layer ensemble averaging, a technique to map pre-trained neural network solutions from software to defective hardware crossbars of emerging memory devices and reliably attain near-software performance on inference. The approach is investigated using a custom 20,000-device hardware prototyping platform on a continual learning problem where a network must learn new tasks without catastrophically forgetting previously learned information. Results demonstrate that by trading off the number of devices required for layer mapping, layer ensemble averaging can reliably boost defective memristive network performance up to the software baseline. For the investigated problem, the average multi-task classification accuracy improves from 61 % to 72 % (< 1 % of software baseline) using the proposed approach.