# Deep physical neural networks enabled by a backpropagation algorithm for arbitrary physical systems

arXiv: https://arxiv.org/abs/2104.13386 · HTML: https://mlsanity.com/paper/2104.13386/

## Key facts

- Cited by 128 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 107 references that are in the map.
- First posted on arXiv 27 Apr 2021.

## Builds on (most cited first, 30 of 33)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [Optuna: A Next-generation Hyperparameter Optimization Framework](https://mlsanity.com/paper/1907.10902/) (2019)
- [Carbon Emissions and Large Neural Network Training](https://mlsanity.com/paper/2104.10350/) (2021)
- [Quantized Neural Networks: Training Neural Networks with Low Precision Weights and Activations](https://mlsanity.com/paper/1609.07061/) (2016)
- [CNN Features off-the-shelf: an Astounding Baseline for Recognition](https://mlsanity.com/paper/1403.6382/) (2014)
- [Equilibrium Propagation: Bridging the Gap Between Energy-Based Models and Backpropagation](https://mlsanity.com/paper/1602.05179/) (2016)
- [All-Optical Machine Learning Using Diffractive Deep Neural Networks](https://mlsanity.com/paper/1804.08711/) (2018)
- [The Hardware Lottery](https://mlsanity.com/paper/2009.06489/) (2020)
- [A Differentiable Physics Engine for Deep Learning in Robotics](https://mlsanity.com/paper/1611.01652/) (2016)
- [Why does deep and cheap learning work so well?](https://mlsanity.com/paper/1608.08225/) (2016)
- [Theoretical Issues in Deep Networks: Approximation, Optimization and Generalization](https://mlsanity.com/paper/1908.09375/) (2019)
- [Scaling Equilibrium Propagation to Deep ConvNets by Drastically Reducing its Gradient Estimator Bias](https://mlsanity.com/paper/2006.03824/) (2020)
- [Direct Feedback Alignment Scales to Modern Deep Learning Tasks and Architectures](https://mlsanity.com/paper/2006.12878/) (2020)
- [Training of photonic neural networks through in situ backpropagation](https://mlsanity.com/paper/1805.09943/) (2018)
- [Wave Physics as an Analog Recurrent Neural Network](https://mlsanity.com/paper/1904.12831/) (2019)
- [A flexible and fast PyTorch toolkit for simulating training and inference on analog crossbar arrays](https://mlsanity.com/paper/2104.02184/) (2021)
- [Surrogate gradients for analog neuromorphic computing](https://mlsanity.com/paper/2006.07239/) (2020)
- [Building high accuracy emulators for scientific simulations with deep neural architecture search](https://mlsanity.com/paper/2001.08055/) (2020)
- [EqSpike: Spike-driven Equilibrium Propagation for Neuromorphic Implementations](https://mlsanity.com/paper/2010.07859/) (2020)
- [Reinforcement Learning in a large scale photonic Recurrent Neural Network](https://mlsanity.com/paper/1711.05133/) (2017)
- [Survey of Machine Learning Accelerators](https://mlsanity.com/paper/2009.00993/) (2020)
- [Self-learning Machines based on Hamiltonian Echo Backpropagation](https://mlsanity.com/paper/2103.04992/) (2021)
- [Equilibrium Propagation with Continual Weight Updates](https://mlsanity.com/paper/2005.04168/) (2020)
- [Lets keep it simple, Using simple architectures to outperform deeper and more complex architectures](https://mlsanity.com/paper/1608.06037/) (2016)
- [Vowel recognition with four coupled spin-torque nano-oscillators](https://mlsanity.com/paper/1711.02704/) (2017)
- [Summarizing CPU and GPU Design Trends with Product Data](https://mlsanity.com/paper/1911.11313/) (2019)
- [Trainable and Dynamic Computing: Error Backpropagation through Physical Media](https://mlsanity.com/paper/1407.6637/) (2014)

## Built on by (most cited first, 30 of 128)

- [Training of Physical Neural Networks](https://mlsanity.com/paper/2406.03372/) (2024)
- [Experimentally realized in situ backpropagation for deep learning in nanophotonic neural networks](https://mlsanity.com/paper/2205.08501/) (2022)
- [Hardware-aware training for large-scale and diverse deep learning inference workloads using in-memory computing-based accelerators](https://mlsanity.com/paper/2302.08469/) (2023)
- [Backpropagation-free Training of Deep Physical Neural Networks](https://mlsanity.com/paper/2304.11042/) (2023)
- [Learning on Arbitrary Graph Topologies via Predictive Coding](https://mlsanity.com/paper/2201.13180/) (2022)
- [Training an Ising Machine with Equilibrium Propagation](https://mlsanity.com/paper/2305.18321/) (2023)
- [Nonlinear Processing with Linear Optics](https://mlsanity.com/paper/2307.08533/) (2023)
- [Machine Learning Without a Processor: Emergent Learning in a Nonlinear Electronic Metamaterial](https://mlsanity.com/paper/2311.00537/) (2023)
- [Forward-Forward Training of an Optical Neural Network](https://mlsanity.com/paper/2305.19170/) (2023)
- [Neural Oscillators are Universal](https://mlsanity.com/paper/2305.08753/) (2023)
- [Hybrid training of optical neural networks](https://mlsanity.com/paper/2203.11207/) (2022)
- [Agnostic Physics-Driven Deep Learning](https://mlsanity.com/paper/2205.15021/) (2022)
- [Thermodynamic AI and the fluctuation frontier](https://mlsanity.com/paper/2302.06584/) (2023)
- [Nonlinear optical encoding enabled by recurrent linear scattering](https://mlsanity.com/paper/2307.08558/) (2023)
- [A compact butterfly-style silicon photonic-electronic neural chip for hardware-efficient deep learning](https://mlsanity.com/paper/2111.06705/) (2021)
- [Learning by non-interfering feedback chemical signaling in physical networks](https://mlsanity.com/paper/2203.12098/) (2022)
- [Frequency propagation: Multi-mechanism learning in nonlinear physical networks](https://mlsanity.com/paper/2208.08862/) (2022)
- [Residual-based Attention Physics-informed Neural Networks for Spatio-Temporal Ageing Assessment of Transformers Operated in Renewable Power Plants](https://mlsanity.com/paper/2405.06443/) (2024)
- [Polarization Multiplexed Diffractive Computing: All-Optical Implementation of a Group of Linear Transformations Through a Polarization-Encoded Diffractive Network](https://mlsanity.com/paper/2203.13482/) (2022)
- [Dual adaptive training of photonic neural networks](https://mlsanity.com/paper/2212.06141/) (2022)
- [Arbitrary control over multimode wave propagation for machine learning](https://mlsanity.com/paper/2402.17750/) (2024)
- [Quantum Equilibrium Propagation for efficient training of quantum systems based on Onsager reciprocity](https://mlsanity.com/paper/2406.06482/) (2024)
- [Multiplexed gradient descent: Fast online training of modern datasets on hardware neural networks without backpropagation](https://mlsanity.com/paper/2303.03986/) (2023)
- [Quantum-limited stochastic optical neural networks operating at a few quanta per activation](https://mlsanity.com/paper/2307.15712/) (2023)
- [Optical Generative Models](https://mlsanity.com/paper/2410.17970/) (2024)
- [Training neural networks with end-to-end optical backpropagation](https://mlsanity.com/paper/2308.05226/) (2023)
- [A Pontryagin Perspective on Reinforcement Learning](https://mlsanity.com/paper/2405.18100/) (2024)
- [Towards Exact Gradient-based Training on Analog In-memory Computing](https://mlsanity.com/paper/2406.12774/) (2024)
- [Towards the Training of Deeper Predictive Coding Neural Networks](https://mlsanity.com/paper/2506.23800/) (2025)
- [A universal approximation theorem for nonlinear resistive networks](https://mlsanity.com/paper/2312.15063/) (2023)

## Abstract (arXiv)

Deep neural networks have become a pervasive tool in science and engineering. However, modern deep neural networks' growing energy requirements now increasingly limit their scaling and broader use. We propose a radical alternative for implementing deep neural network models: Physical Neural Networks. We introduce a hybrid physical-digital algorithm called Physics-Aware Training to efficiently train sequences of controllable physical systems to act as deep neural networks. This method automatically trains the functionality of any sequence of real physical systems, directly, using backpropagation, the same technique used for modern deep neural networks. To illustrate their generality, we demonstrate physical neural networks with three diverse physical systems-optical, mechanical, and electrical. Physical neural networks may facilitate unconventional machine learning hardware that is orders of magnitude faster and more energy efficient than conventional electronic processors.