# Quantum-limited stochastic optical neural networks operating at a few quanta per activation

arXiv: https://arxiv.org/abs/2307.15712 · HTML: https://mlsanity.com/paper/2307.15712/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 84 references that are in the map.
- First posted on arXiv 28 Jul 2023.

## Builds on (most cited first, 25 of 25)

- [Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation](https://mlsanity.com/paper/1308.3432/) (2013)
- [DoReFa-Net: Training Low Bitwidth Convolutional Neural Networks with Low Bitwidth Gradients](https://mlsanity.com/paper/1606.06160/) (2016)
- [Understanding Straight-Through Estimator in Training Activation Quantized Neural Nets](https://mlsanity.com/paper/1903.05662/) (2019)
- [Challenges and Opportunities in Quantum Machine Learning](https://mlsanity.com/paper/2303.09491/) (2023)
- [Binary Neural Networks: A Survey](https://mlsanity.com/paper/2004.03333/) (2020)
- [All-Optical Machine Learning Using Diffractive Deep Neural Networks](https://mlsanity.com/paper/1804.08711/) (2018)
- [Deep physical neural networks enabled by a backpropagation algorithm for arbitrary physical systems](https://mlsanity.com/paper/2104.13386/) (2021)
- [An Analysis of Deep Neural Network Models for Practical Applications](https://mlsanity.com/paper/1605.07678/) (2016)
- [Convolutional Networks for Fast, Energy-Efficient Neuromorphic Computing](https://mlsanity.com/paper/1603.08270/) (2016)
- [Large-scale neuromorphic optoelectronic computing with a reconfigurable diffractive processing unit](https://mlsanity.com/paper/2008.11659/) (2020)
- [Wave Physics as an Analog Recurrent Neural Network](https://mlsanity.com/paper/1904.12831/) (2019)
- [Surrogate gradients for analog neuromorphic computing](https://mlsanity.com/paper/2006.07239/) (2020)
- [Generalizing Pooling Functions in Convolutional Neural Networks: Mixed, Gated, and Tree](https://mlsanity.com/paper/1509.08985/) (2015)
- [An optical neural network using less than 1 photon per multiplication](https://mlsanity.com/paper/2104.13467/) (2021)
- [Vowel recognition with four coupled spin-torque nano-oscillators](https://mlsanity.com/paper/1711.02704/) (2017)
- [Massively Parallel Amplitude-Only Fourier Neural Network](https://mlsanity.com/paper/2008.05853/) (2020)
- [Noisy Machines: Understanding Noisy Neural Networks and Enhancing Robustness to Analog Hardware Errors Using Distillation](https://mlsanity.com/paper/2001.04974/) (2020)
- [Single-Shot Optical Neural Network](https://mlsanity.com/paper/2205.09103/) (2022)
- [Improving Noise Tolerance of Mixed-Signal Neural Networks](https://mlsanity.com/paper/1904.01705/) (2019)
- [Understanding and mitigating noise in trained deep neural networks](https://mlsanity.com/paper/2103.07413/) (2021)
- [ASP Vision: Optically Computing the First Layer of Convolutional Neural Networks using Angle Sensitive Pixels](https://mlsanity.com/paper/1605.03621/) (2016)
- [Energy-Efficient Hybrid Stochastic-Binary Neural Networks for Near-Sensor Computing](https://mlsanity.com/paper/1706.02344/) (2017)
- [Noise mitigation strategies in physical feedforward neural networks](https://mlsanity.com/paper/2204.09461/) (2022)
- [RF-Photonic Deep Learning Processor with Shannon-Limited Data Movement](https://mlsanity.com/paper/2207.06883/) (2022)
- [How to do Physics-based Learning](https://mlsanity.com/paper/2005.13531/) (2020)

## Built on by (most cited first, 8 of 8)

- [Training of Physical Neural Networks](https://mlsanity.com/paper/2406.03372/) (2024)
- [Stochastic Reservoir Computers](https://mlsanity.com/paper/2405.12382/) (2024)
- [Photonics-Enhanced Graph Convolutional Networks](https://mlsanity.com/paper/2512.15549/) (2025)
- [Quantum-secure multiparty deep learning](https://mlsanity.com/paper/2408.05629/) (2024)
- [Low-rank surrogate modeling and stochastic zero-order optimization for training of neural networks with black-box layers](https://mlsanity.com/paper/2509.15113/) (2025)
- [Physical Foundation Models: Fixed hardware implementations of large-scale neural networks](https://mlsanity.com/paper/2604.27911/) (2026)
- [Impact of white Gaussian internal noise on analog echo-state neural networks](https://mlsanity.com/paper/2405.07670/) (2024)
- [Measurement-Adapted Eigentask Representations for Photon-Limited Optical Readout](https://mlsanity.com/paper/2605.10008/) (2026)

## Abstract (arXiv)

Energy efficiency in computation is ultimately limited by noise, with quantum limits setting the fundamental noise floor. Analog physical neural networks hold promise for improved energy efficiency compared to digital electronic neural networks. However, they are typically operated in a relatively high-power regime so that the signal-to-noise ratio (SNR) is large, and the noise can be treated as a perturbation. We study optical neural networks where all layers except the last are operated in the limit that each neuron can be activated by just a single photon, and as a result the noise on neuron activations is no longer merely perturbative. We show that by using a physics-based probabilistic model of the neuron activations in training, it is possible to perform accurate machine-learning inference in spite of the extremely high shot noise (SNR ~ 1). We experimentally demonstrated MNIST handwritten-digit classification with a test accuracy of 98% using an optical neural network with a hidden layer operating in the single-photon regime; the optical energy used to perform the classification corresponds to just 0.038 photons per multiply-accumulate (MAC) operation. Our physics-aware stochastic training approach might also prove useful with non-optical ultra-low-power hardware.