# Backpropagation through nonlinear units for all-optical training of neural networks

arXiv: https://arxiv.org/abs/1912.12256 · HTML: https://mlsanity.com/paper/1912.12256/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Dec 2019.

## Built on by (most cited first, 2 of 2)

- [Photonic Differential Privacy with Direct Feedback Alignment](https://mlsanity.com/paper/2106.03645/) (2021)
- [Training Large-Scale Optical Neural Networks with Two-Pass Forward Propagation](https://mlsanity.com/paper/2408.08337/) (2024)

## Abstract (arXiv)

Backpropagation through nonlinear neurons is an outstanding challenge to the field of optical neural networks and the major conceptual barrier to all-optical training schemes. Each neuron is required to exhibit a directionally dependent response to propagating optical signals, with the backwards response conditioned on the forward signal, which is highly non-trivial to implement optically. We propose a practical and surprisingly simple solution that uses saturable absorption to provide the network nonlinearity. We find that the backward propagating gradients required to train the network can be approximated in a pump-probe scheme that requires only passive optical elements. Simulations show that, with readily obtainable optical depths, our approach can achieve equivalent performance to state-of-the-art computational networks on image classification benchmarks, even in deep networks with multiple sequential gradient approximations. This scheme is compatible with leading optical neural network proposals and therefore provides a feasible path towards end-to-end optical training.