# Multiplexed gradient descent: Fast online training of modern datasets on hardware neural networks without backpropagation

arXiv: https://arxiv.org/abs/2303.03986 · HTML: https://mlsanity.com/paper/2303.03986/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 60 references that are in the map.
- First posted on arXiv 5 Mar 2023.

## Builds on (most cited first, 12 of 12)

- [Equilibrium Propagation: Bridging the Gap Between Energy-Based Models and Backpropagation](https://mlsanity.com/paper/1602.05179/) (2016)
- [The Computational Limits of Deep Learning](https://mlsanity.com/paper/2007.05558/) (2020)
- [Deep physical neural networks enabled by a backpropagation algorithm for arbitrary physical systems](https://mlsanity.com/paper/2104.13386/) (2021)
- [A Survey of Neuromorphic Computing and Neural Networks in Hardware](https://mlsanity.com/paper/1705.06963/) (2017)
- [Neuromorphic Silicon Photonic Networks](https://mlsanity.com/paper/1611.02272/) (2016)
- [Eligibility Traces and Plasticity on Behavioral Time Scales: Experimental Support of neoHebbian Three-Factor Learning Rules](https://mlsanity.com/paper/1801.05219/) (2018)
- [Large-scale neuromorphic optoelectronic computing with a reconfigurable diffractive processing unit](https://mlsanity.com/paper/2008.11659/) (2020)
- [Scaling Forward Gradient With Local Losses](https://mlsanity.com/paper/2210.03310/) (2022)
- [Brain-inspired computing: We need a master plan](https://mlsanity.com/paper/2104.14517/) (2021)
- [Brain-Inspired Learning on Neuromorphic Substrates](https://mlsanity.com/paper/2010.11931/) (2020)
- [Design of optical neural networks with component imprecisions](https://mlsanity.com/paper/2001.01681/) (2019)
- [Short-term plasticity as cause-effect hypothesis testing in distal reward learning](https://mlsanity.com/paper/1402.0710/) (2014)

## Built on by (most cited first, 8 of 8)

- [Training of Physical Neural Networks](https://mlsanity.com/paper/2406.03372/) (2024)
- [Training Convolutional Neural Networks with the Forward-Forward algorithm](https://mlsanity.com/paper/2312.14924/) (2023)
- [Model-free Optical Processors using In Situ Reinforcement Learning with Proximal Policy Optimization](https://mlsanity.com/paper/2507.05583/) (2025)
- [Low-rank surrogate modeling and stochastic zero-order optimization for training of neural networks with black-box layers](https://mlsanity.com/paper/2509.15113/) (2025)
- [Self-training superconducting neuromorphic circuits using reinforcement learning rules](https://mlsanity.com/paper/2404.18774/) (2024)
- [Scaling of hardware-compatible perturbative training algorithms](https://mlsanity.com/paper/2501.15403/) (2025)
- [BOND: License to Train with Black-Box Functions](https://mlsanity.com/paper/2510.25074/) (2025)
- [Analog Physical Systems Can Exhibit Double Descent](https://mlsanity.com/paper/2511.17825/) (2025)

## Abstract (arXiv)

We present multiplexed gradient descent (MGD), a gradient descent framework designed to easily train analog or digital neural networks in hardware. MGD utilizes zero-order optimization techniques for online training of hardware neural networks. We demonstrate its ability to train neural networks on modern machine learning datasets, including CIFAR-10 and Fashion-MNIST, and compare its performance to backpropagation. Assuming realistic timescales and hardware parameters, our results indicate that these optimization techniques can train a network on emerging hardware platforms orders of magnitude faster than the wall-clock time of training via backpropagation on a standard GPU, even in the presence of imperfect weight updates or device-to-device variations in the hardware. We additionally describe how it can be applied to existing hardware as part of chip-in-the-loop training, or integrated directly at the hardware level. Crucially, the MGD framework is highly flexible, and its gradient descent process can be optimized to compensate for specific hardware limitations such as slow parameter-update speeds or limited input bandwidth.