# An optical neural network using less than 1 photon per multiplication

arXiv: https://arxiv.org/abs/2104.13467 · HTML: https://mlsanity.com/paper/2104.13467/

## Key facts

- Cited by 30 papers in the map (data through 14 Oct 2026).
- Builds on 16 of its 77 references that are in the map.
- First posted on arXiv 27 Apr 2021.

## Builds on (most cited first, 16 of 16)

- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Optuna: A Next-generation Hyperparameter Optimization Framework](https://mlsanity.com/paper/1907.10902/) (2019)
- [Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference](https://mlsanity.com/paper/1712.05877/) (2017)
- [In-Datacenter Performance Analysis of a Tensor Processing Unit](https://mlsanity.com/paper/1704.04760/) (2017)
- [Efficient Processing of Deep Neural Networks: A Tutorial and Survey](https://mlsanity.com/paper/1703.09039/) (2017)
- [Deep Learning with Limited Numerical Precision](https://mlsanity.com/paper/1502.02551/) (2015)
- [The Computational Limits of Deep Learning](https://mlsanity.com/paper/2007.05558/) (2020)
- [All-Optical Machine Learning Using Diffractive Deep Neural Networks](https://mlsanity.com/paper/1804.08711/) (2018)
- [Photonics for artificial intelligence and neuromorphic computing](https://mlsanity.com/paper/2011.00111/) (2020)
- [Large-scale neuromorphic optoelectronic computing with a reconfigurable diffractive processing unit](https://mlsanity.com/paper/2008.11659/) (2020)
- [Taming the Wild: A Unified Analysis of Hogwild!-Style Algorithms](https://mlsanity.com/paper/1506.06438/) (2015)
- [Reinforcement Learning in a large scale photonic Recurrent Neural Network](https://mlsanity.com/paper/1711.05133/) (2017)
- [Survey of Machine Learning Accelerators](https://mlsanity.com/paper/2009.00993/) (2020)
- [Massively Parallel Amplitude-Only Fourier Neural Network](https://mlsanity.com/paper/2008.05853/) (2020)
- [BinarEye: An Always-On Energy-Accuracy-Scalable Binary CNN Processor With All Memory On Chip in 28nm CMOS](https://mlsanity.com/paper/1804.05554/) (2018)
- [Understanding and mitigating noise in trained deep neural networks](https://mlsanity.com/paper/2103.07413/) (2021)

## Built on by (most cited first, 30 of 30)

- [Training of Physical Neural Networks](https://mlsanity.com/paper/2406.03372/) (2024)
- [Backpropagation-free Training of Deep Physical Neural Networks](https://mlsanity.com/paper/2304.11042/) (2023)
- [Image sensing with multilayer, nonlinear optical neural networks](https://mlsanity.com/paper/2207.14293/) (2022)
- [Hybrid training of optical neural networks](https://mlsanity.com/paper/2203.11207/) (2022)
- [Single-Shot Optical Neural Network](https://mlsanity.com/paper/2205.09103/) (2022)
- [Optical Transformers](https://mlsanity.com/paper/2302.10360/) (2023)
- [Photonic Advantage of Optical Encoders](https://mlsanity.com/paper/2305.01743/) (2023)
- [Quantum-limited stochastic optical neural networks operating at a few quanta per activation](https://mlsanity.com/paper/2307.15712/) (2023)
- [Intelligent Computing: The Latest Advances, Challenges and Future](https://mlsanity.com/paper/2211.11281/) (2022)
- [Training all-mechanical neural networks for task learning through in situ backpropagation](https://mlsanity.com/paper/2404.15471/) (2024)
- [Noise mitigation strategies in physical feedforward neural networks](https://mlsanity.com/paper/2204.09461/) (2022)
- [Privacy-Enhancing Optical Embeddings for Lensless Classification](https://mlsanity.com/paper/2211.12864/) (2022)
- [RF-Photonic Deep Learning Processor with Shannon-Limited Data Movement](https://mlsanity.com/paper/2207.06883/) (2022)
- [AnalogVNN: A fully modular framework for modeling and optimizing photonic neural networks](https://mlsanity.com/paper/2210.10048/) (2022)
- [Photonics-Enhanced Graph Convolutional Networks](https://mlsanity.com/paper/2512.15549/) (2025)
- [Training a neural network with exciton-polariton optical nonlinearity](https://mlsanity.com/paper/2107.11156/) (2021)
- [High Throughput Multi-Channel Parallelized Diffraction Convolutional Neural Network Accelerator](https://mlsanity.com/paper/2112.12297/) (2021)
- [Quantum-secure multiparty deep learning](https://mlsanity.com/paper/2408.05629/) (2024)
- [Impact of white noise in artificial neural networks trained for classification: performance and noise mitigation strategies](https://mlsanity.com/paper/2411.04354/) (2024)
- [Scientific Computing with Diffractive Optical Neural Networks](https://mlsanity.com/paper/2302.10905/) (2023)
- [Intelligent Multi-channel Meta-imagers for Accelerating Machine Vision](https://mlsanity.com/paper/2306.07365/) (2023)
- [Hyperspectral In-Memory Computing with Optical Frequency Combs and Programmable Optical Memories](https://mlsanity.com/paper/2310.11014/) (2023)
- [LUM-ViT: Learnable Under-sampling Mask Vision Transformer for Bandwidth Limited Optical Signal Acquisition](https://mlsanity.com/paper/2403.01412/) (2024)
- [Impact of white Gaussian internal noise on analog echo-state neural networks](https://mlsanity.com/paper/2405.07670/) (2024)
- [Automating Experimental Optics with Sample Efficient Machine Learning Methods](https://mlsanity.com/paper/2503.14260/) (2025)
- [Optical Echo State Network Reservoir Computing](https://mlsanity.com/paper/2504.08224/) (2025)
- [Machine Intelligence on Wireless Edge Networks](https://mlsanity.com/paper/2506.12210/) (2025)
- [Laser interferometry as a robust neuromorphic platform for machine learning](https://mlsanity.com/paper/2601.18047/) (2026)
- [Extraction of linearized models from pre-trained networks via knowledge distillation](https://mlsanity.com/paper/2604.06732/) (2026)
- [Training single-electron and single-photon stochastic physical neural networks](https://mlsanity.com/paper/2604.10861/) (2026)

## Abstract (arXiv)

Deep learning has rapidly become a widespread tool in both scientific and commercial endeavors. Milestones of deep learning exceeding human performance have been achieved for a growing number of tasks over the past several years, across areas as diverse as game-playing, natural-language translation, and medical-image analysis. However, continued progress is increasingly hampered by the high energy costs associated with training and running deep neural networks on electronic processors. Optical neural networks have attracted attention as an alternative physical platform for deep learning, as it has been theoretically predicted that they can fundamentally achieve higher energy efficiency than neural networks deployed on conventional digital computers. Here, we experimentally demonstrate an optical neural network achieving 99% accuracy on handwritten-digit classification using ~3.2 detected photons per weight multiplication and ~90% accuracy using ~0.64 photons (~2.4 × 10-19 J of optical energy) per weight multiplication. This performance was achieved using a custom free-space optical processor that executes matrix-vector multiplications in a massively parallel fashion, with up to ~0.5 million scalar (weight) multiplications performed at the same time. Using commercially available optical components and standard neural-network training methods, we demonstrated that optical neural networks can operate near the standard quantum limit with extremely low optical powers and still achieve high accuracy. Our results provide a proof-of-principle for low-optical-power operation, and with careful system design including the surrounding electronics used for data storage and control, open up a path to realizing optical processors that require only 10-16 J total energy per scalar multiplication -- which is orders of magnitude more efficient than current digital processors.