# Backpropagation-free Training of Deep Physical Neural Networks

arXiv: https://arxiv.org/abs/2304.11042 · HTML: https://mlsanity.com/paper/2304.11042/

## Key facts

- Cited by 42 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 87 references that are in the map.
- First posted on arXiv 20 Apr 2023.

## Builds on (most cited first, 20 of 20)

- [Carbon Emissions and Large Neural Network Training](https://mlsanity.com/paper/2104.10350/) (2021)
- [On the Number of Linear Regions of Deep Neural Networks](https://mlsanity.com/paper/1402.1869/) (2014)
- [The Forward-Forward Algorithm: Some Preliminary Investigations](https://mlsanity.com/paper/2212.13345/) (2022)
- [Equilibrium Propagation: Bridging the Gap Between Energy-Based Models and Backpropagation](https://mlsanity.com/paper/1602.05179/) (2016)
- [All-Optical Machine Learning Using Diffractive Deep Neural Networks](https://mlsanity.com/paper/1804.08711/) (2018)
- [Deep physical neural networks enabled by a backpropagation algorithm for arbitrary physical systems](https://mlsanity.com/paper/2104.13386/) (2021)
- [Training of photonic neural networks through in situ backpropagation](https://mlsanity.com/paper/1805.09943/) (2018)
- [Experimentally realized in situ backpropagation for deep learning in nanophotonic neural networks](https://mlsanity.com/paper/2205.08501/) (2022)
- [Wave Physics as an Analog Recurrent Neural Network](https://mlsanity.com/paper/1904.12831/) (2019)
- [Reinforcement Learning in a large scale photonic Recurrent Neural Network](https://mlsanity.com/paper/1711.05133/) (2017)
- [An optical neural network using less than 1 photon per multiplication](https://mlsanity.com/paper/2104.13467/) (2021)
- [SymBa: Symmetric Backpropagation-Free Contrastive Learning with Forward-Forward Algorithm for Optimizing Convergence](https://mlsanity.com/paper/2303.08418/) (2023)
- [Vowel recognition with four coupled spin-torque nano-oscillators](https://mlsanity.com/paper/1711.02704/) (2017)
- [Blockwise Self-Supervised Learning at Scale](https://mlsanity.com/paper/2302.01647/) (2023)
- [Scalable Optical Learning Operator](https://mlsanity.com/paper/2012.12404/) (2020)
- [Interlocking Backpropagation: Improving depthwise model-parallelism](https://mlsanity.com/paper/2010.04116/) (2020)
- [Forward Learning with Top-Down Feedback: Empirical and Analytical Characterization](https://mlsanity.com/paper/2302.05440/) (2023)
- [Wave-based extreme deep learning based on non-linear time-Floquet entanglement](https://mlsanity.com/paper/2107.08564/) (2021)
- [On the Tacit Linearity Assumption in Common Cascaded Models of RIS-Parametrized Wireless Channels](https://mlsanity.com/paper/2302.04993/) (2023)
- [Physics-inspired Neuroacoustic Computing Based on Tunable Nonlinear Multiple-scattering](https://mlsanity.com/paper/2304.08380/) (2023)

## Built on by (most cited first, 30 of 42)

- [Training of Physical Neural Networks](https://mlsanity.com/paper/2406.03372/) (2024)
- [Training an Ising Machine with Equilibrium Propagation](https://mlsanity.com/paper/2305.18321/) (2023)
- [Forward-Forward Training of an Optical Neural Network](https://mlsanity.com/paper/2305.19170/) (2023)
- [Self-Contrastive Forward-Forward Algorithm](https://mlsanity.com/paper/2409.11593/) (2024)
- [Nonlinear optical encoding enabled by recurrent linear scattering](https://mlsanity.com/paper/2307.08558/) (2023)
- [Quantum Equilibrium Propagation for efficient training of quantum systems based on Onsager reciprocity](https://mlsanity.com/paper/2406.06482/) (2024)
- [A universal approximation theorem for nonlinear resistive networks](https://mlsanity.com/paper/2312.15063/) (2023)
- [Ornstein-Uhlenbeck Adaptation as a Mechanism for Learning in Brains and Machines](https://mlsanity.com/paper/2410.13563/) (2024)
- [Noise-Aware Training of Neuromorphic Dynamic Device Networks](https://mlsanity.com/paper/2401.07387/) (2024)
- [End-to-end autonomous scientific discovery on a real optical platform](https://mlsanity.com/paper/2604.27092/) (2026)
- [Towards training digitally-tied analog blocks via hybrid gradient computation](https://mlsanity.com/paper/2409.03306/) (2024)
- [FLOPS: Forward Learning with OPtimal Sampling](https://mlsanity.com/paper/2410.05966/) (2024)
- [Asymmetrical estimator for training encapsulated deep photonic neural networks](https://mlsanity.com/paper/2405.18458/) (2024)
- [Iterative Quantum Feature Maps](https://mlsanity.com/paper/2506.19461/) (2025)
- [Near-Equilibrium Propagation training in nonlinear wave systems](https://mlsanity.com/paper/2510.16084/) (2025)
- [Training Spiking Neural Networks via Augmented Direct Feedback Alignment](https://mlsanity.com/paper/2409.07776/) (2024)
- [Noise-based Local Learning using Stochastic Magnetic Tunnel Junctions](https://mlsanity.com/paper/2412.12783/) (2024)
- [Blending Optimal Control and Biologically Plausible Learning for Noise-Robust Physical Neural Networks](https://mlsanity.com/paper/2502.19053/) (2025)
- [Nonlinear Computation with Linear Optics via Source-Position Encoding](https://mlsanity.com/paper/2504.20401/) (2025)
- [Model-free Optical Processors using In Situ Reinforcement Learning with Proximal Policy Optimization](https://mlsanity.com/paper/2507.05583/) (2025)
- [Function regression using the forward forward training and inferring paradigm](https://mlsanity.com/paper/2510.06762/) (2025)
- [Learning Using a Single Forward Pass](https://mlsanity.com/paper/2402.09769/) (2024)
- [Harnessing Nonidealities in Analog In-Memory Computing Circuits: A Physical Modeling Approach for Neuromorphic Systems](https://mlsanity.com/paper/2412.09010/) (2024)
- [One-Time Soft Alignment Enables Resilient Learning without Weight Transport](https://mlsanity.com/paper/2505.20892/) (2025)
- [Training deep physical neural networks with local physical information bottleneck](https://mlsanity.com/paper/2602.09569/) (2026)
- [Local learning for stable backpropagation-free neural network training towards physical learning](https://mlsanity.com/paper/2603.24790/) (2026)
- [Physical Foundation Models: Fixed hardware implementations of large-scale neural networks](https://mlsanity.com/paper/2604.27911/) (2026)
- [Thermal Crosstalk Modelling and Compensation Methods for Programmable Photonic Integrated Circuits](https://mlsanity.com/paper/2404.10589/) (2024)
- [Generalizing Deep Surrogate Solvers for Broadband Electromagnetic Field Prediction at Unseen Wavelengths](https://mlsanity.com/paper/2408.02971/) (2024)
- [Thermodynamic cost of inference and learning in physical neural networks](https://mlsanity.com/paper/2503.09980/) (2025)

## Abstract (arXiv)

Recent years have witnessed the outstanding success of deep learning in various fields such as vision and natural language processing. This success is largely indebted to the massive size of deep learning models that is expected to increase unceasingly. This growth of the deep learning models is accompanied by issues related to their considerable energy consumption, both during the training and inference phases, as well as their scalability. Although a number of work based on unconventional physical systems have been proposed which addresses the issue of energy efficiency in the inference phase, efficient training of deep learning models has remained unaddressed. So far, training of digital deep learning models mainly relies on backpropagation, which is not suitable for physical implementation as it requires perfect knowledge of the computation performed in the so-called forward pass of the neural network. Here, we tackle this issue by proposing a simple deep neural network architecture augmented by a biologically plausible learning algorithm, referred to as "model-free forward-forward training". The proposed architecture enables training deep physical neural networks consisting of layers of physical nonlinear systems, without requiring detailed knowledge of the nonlinear physical layers' properties. We show that our method outperforms state-of-the-art hardware-aware training methods by improving training speed, decreasing digital computations, and reducing power consumption in physical systems. We demonstrate the adaptability of the proposed method, even in systems exposed to dynamic or unpredictable external perturbations. To showcase the universality of our approach, we train diverse wave-based physical neural networks that vary in the underlying wave phenomenon and the type of non-linearity they use, to perform vowel and image classification tasks experimentally.