# Experimentally realized in situ backpropagation for deep learning in nanophotonic neural networks

arXiv: https://arxiv.org/abs/2205.08501 · HTML: https://mlsanity.com/paper/2205.08501/

## Key facts

- Cited by 44 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 72 references that are in the map.
- First posted on arXiv 17 May 2022.

## Builds on (most cited first, 8 of 8)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Direct Feedback Alignment Provides Learning in Deep Neural Networks](https://mlsanity.com/paper/1609.01596/) (2016)
- [Deep physical neural networks enabled by a backpropagation algorithm for arbitrary physical systems](https://mlsanity.com/paper/2104.13386/) (2021)
- [Training of photonic neural networks through in situ backpropagation](https://mlsanity.com/paper/1805.09943/) (2018)
- [Reprogrammable Electro-Optic Nonlinear Activation Functions for Optical Neural Networks](https://mlsanity.com/paper/1903.04579/) (2019)
- [Silicon Photonic Architecture for Training Deep Neural Networks with Direct Feedback Alignment](https://mlsanity.com/paper/2111.06862/) (2021)
- [Hybrid training of optical neural networks](https://mlsanity.com/paper/2203.11207/) (2022)
- [Parallel fault-tolerant programming of an arbitrary feedforward photonic network](https://mlsanity.com/paper/1909.06179/) (2019)

## Built on by (most cited first, 30 of 44)

- [Training of Physical Neural Networks](https://mlsanity.com/paper/2406.03372/) (2024)
- [Backpropagation-free Training of Deep Physical Neural Networks](https://mlsanity.com/paper/2304.11042/) (2023)
- [Spatially Varying Nanophotonic Neural Networks](https://mlsanity.com/paper/2308.03407/) (2023)
- [Arbitrary control over multimode wave propagation for machine learning](https://mlsanity.com/paper/2402.17750/) (2024)
- [Quantum Equilibrium Propagation for efficient training of quantum systems based on Onsager reciprocity](https://mlsanity.com/paper/2406.06482/) (2024)
- [Training neural networks with end-to-end optical backpropagation](https://mlsanity.com/paper/2308.05226/) (2023)
- [M3ICRO: Machine Learning-Enabled Compact Photonic Tensor Core based on PRogrammable Multi-Operand Multimode Interference](https://mlsanity.com/paper/2305.19505/) (2023)
- [Training all-mechanical neural networks for task learning through in situ backpropagation](https://mlsanity.com/paper/2404.15471/) (2024)
- [Poor Man's Training on MCUs: A Memory-Efficient Quantized Back-Propagation-Free Approach](https://mlsanity.com/paper/2411.05873/) (2024)
- [Real-Time FJ/MAC PDE Solvers via Tensorized, Back-Propagation-Free Optical PINN Training](https://mlsanity.com/paper/2401.00413/) (2023)
- [A Blueprint for Precise and Fault-Tolerant Analog Neural Networks](https://mlsanity.com/paper/2309.10759/) (2023)
- [Asymmetrical estimator for training encapsulated deep photonic neural networks](https://mlsanity.com/paper/2405.18458/) (2024)
- [Near-Equilibrium Propagation training in nonlinear wave systems](https://mlsanity.com/paper/2510.16084/) (2025)
- [Mirage: An RNS-Based Photonic Accelerator for DNN Training](https://mlsanity.com/paper/2311.17323/) (2023)
- [Multiplexed all-optical permutation operations using a reconfigurable diffractive optical network](https://mlsanity.com/paper/2402.02397/) (2024)
- [Scalable Back-Propagation-Free Training of Optical Physics-Informed Neural Networks](https://mlsanity.com/paper/2502.12384/) (2025)
- [Model-free Optical Processors using In Situ Reinforcement Learning with Proximal Policy Optimization](https://mlsanity.com/paper/2507.05583/) (2025)
- [DOCTOR: Dynamic On-Chip Temporal Variation Remediation Toward Self-Corrected Photonic Tensor Accelerators](https://mlsanity.com/paper/2403.02688/) (2024)
- [Quantum-secure multiparty deep learning](https://mlsanity.com/paper/2408.05629/) (2024)
- [A ghost mechanism: An analytical model of abrupt learning in recurrent networks](https://mlsanity.com/paper/2501.02378/) (2025)
- [SP2RINT: Spatially-Decoupled Physics-Inspired Progressive Inverse Optimization for Scalable, PDE-Constrained Meta-Optical Neural Network Training](https://mlsanity.com/paper/2505.18377/) (2025)
- [Circuit realization and hardware linearization of monotone operator equilibrium networks](https://mlsanity.com/paper/2509.13793/) (2025)
- [Low-rank surrogate modeling and stochastic zero-order optimization for training of neural networks with black-box layers](https://mlsanity.com/paper/2509.15113/) (2025)
- [Training deep physical neural networks with local physical information bottleneck](https://mlsanity.com/paper/2602.09569/) (2026)
- [Unlocked Backpropagation using Wave Scattering](https://mlsanity.com/paper/2602.10461/) (2026)
- [Local learning for stable backpropagation-free neural network training towards physical learning](https://mlsanity.com/paper/2603.24790/) (2026)
- [Thermal Crosstalk Modelling and Compensation Methods for Programmable Photonic Integrated Circuits](https://mlsanity.com/paper/2404.10589/) (2024)
- [Computational metaoptics for imaging](https://mlsanity.com/paper/2411.09133/) (2024)
- [Experimental Demonstration of an Optical Neural PDE Solver via On-Chip PINN Training](https://mlsanity.com/paper/2501.00742/) (2025)
- [Scaling of hardware-compatible perturbative training algorithms](https://mlsanity.com/paper/2501.15403/) (2025)

## Abstract (arXiv)

Neural networks are widely deployed models across many scientific disciplines and commercial endeavors ranging from edge computing and sensing to large-scale signal processing in data centers. The most efficient and well-entrenched method to train such networks is backpropagation, or reverse-mode automatic differentiation. To counter an exponentially increasing energy budget in the artificial intelligence sector, there has been recent interest in analog implementations of neural networks, specifically nanophotonic neural networks for which no analog backpropagation demonstration exists. We design mass-manufacturable silicon photonic neural networks that alternately cascade our custom designed "photonic mesh" accelerator with digitally implemented nonlinearities. These reconfigurable photonic meshes program computationally intensive arbitrary matrix multiplication by setting physical voltages that tune the interference of optically encoded input data propagating through integrated Mach-Zehnder interferometer networks. Here, using our packaged photonic chip, we demonstrate in situ backpropagation for the first time to solve classification tasks and evaluate a new protocol to keep the entire gradient measurement and update of physical device voltages in the analog domain, improving on past theoretical proposals. Our method is made possible by introducing three changes to typical photonic meshes: (1) measurements at optical "grating tap" monitors, (2) bidirectional optical signal propagation automated by fiber switch, and (3) universal generation and readout of optical amplitude and phase. After training, our classification achieves accuracies similar to digital equivalents even in presence of systematic error. Our findings suggest a new training paradigm for photonics-accelerated artificial intelligence based entirely on a physical analog of the popular backpropagation technique.