# Physics-aware Differentiable Discrete Codesign for Diffractive Optical Neural Networks

arXiv: https://arxiv.org/abs/2209.14252 · HTML: https://mlsanity.com/paper/2209.14252/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 46 references that are in the map.
- First posted on arXiv 28 Sep 2022.

## Builds on (most cited first, 23 of 23)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [Categorical Reparameterization with Gumbel-Softmax](https://mlsanity.com/paper/1611.01144/) (2016)
- [Deep Compression: Compressing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding](https://mlsanity.com/paper/1510.00149/) (2015)
- [The Concrete Distribution: A Continuous Relaxation of Discrete Random Variables](https://mlsanity.com/paper/1611.00712/) (2016)
- [In-Datacenter Performance Analysis of a Tensor Processing Unit](https://mlsanity.com/paper/1704.04760/) (2017)
- [Quantizing deep convolutional networks for efficient inference: A whitepaper](https://mlsanity.com/paper/1806.08342/) (2018)
- [FBNet: Hardware-Aware Efficient ConvNet Design via Differentiable Neural Architecture Search](https://mlsanity.com/paper/1812.03443/) (2018)
- [vq-wav2vec: Self-Supervised Learning of Discrete Speech Representations](https://mlsanity.com/paper/1910.05453/) (2019)
- [All-Optical Machine Learning Using Diffractive Deep Neural Networks](https://mlsanity.com/paper/1804.08711/) (2018)
- [Mixed Precision Quantization of ConvNets via Differentiable Neural Architecture Search](https://mlsanity.com/paper/1812.00090/) (2018)
- [Soft Weight-Sharing for Neural Network Compression](https://mlsanity.com/paper/1702.04008/) (2017)
- [Neuromorphic Silicon Photonic Networks](https://mlsanity.com/paper/1611.02272/) (2016)
- [Large-scale neuromorphic optoelectronic computing with a reconfigurable diffractive processing unit](https://mlsanity.com/paper/2008.11659/) (2020)
- [MiLeNAS: Efficient Neural Architecture Search via Mixed-Level Reformulation](https://mlsanity.com/paper/2003.12238/) (2020)
- [Analysis of Diffractive Optical Neural Networks and Their Integration with Electronic Neural Networks](https://mlsanity.com/paper/1810.01916/) (2018)
- [Efficient On-Chip Learning for Optical Neural Networks Through Power-Aware Sparse Zeroth-Order Optimization](https://mlsanity.com/paper/2012.11148/) (2020)
- [Ensemble learning of diffractive optical networks](https://mlsanity.com/paper/2009.06869/) (2020)
- [Scale-, shift- and rotation-invariant diffractive optical networks](https://mlsanity.com/paper/2010.12747/) (2020)
- [Auto-NBA: Efficient and Effective Search Over the Joint Space of Networks, Bitwidths, and Accelerators](https://mlsanity.com/paper/2106.06575/) (2021)
- [O-HAS: Optical Hardware Accelerator Search for Boosting Both Acceleration Performance and Development Speed](https://mlsanity.com/paper/2108.07538/) (2021)
- [Real-time Multi-Task Diffractive Deep Neural Networks via Hardware-Software Co-design](https://mlsanity.com/paper/2012.08906/) (2020)
- [A3C-S: Automated Agent Accelerator Co-Search towards Efficient Deep Reinforcement Learning](https://mlsanity.com/paper/2106.06577/) (2021)

## Built on by (most cited first, 5 of 5)

- [Rubik's Optical Neural Networks: Multi-task Learning with Physics-aware Rotation Architecture](https://mlsanity.com/paper/2304.12985/) (2023)
- [Physics-aware Roughness Optimization for Diffractive Optical Neural Networks](https://mlsanity.com/paper/2304.01500/) (2023)
- [SP2RINT: Spatially-Decoupled Physics-Inspired Progressive Inverse Optimization for Scalable, PDE-Constrained Meta-Optical Neural Network Training](https://mlsanity.com/paper/2505.18377/) (2025)
- [QuATON: Quantization Aware Training of Optical Neurons](https://mlsanity.com/paper/2310.03049/) (2023)
- [Exploring Deep-to-Shallow Transformable Neural Networks for Intelligent Embedded Systems](https://mlsanity.com/paper/2512.19731/) (2025)

## Abstract (arXiv)

Diffractive optical neural networks (DONNs) have attracted lots of attention as they bring significant advantages in terms of power efficiency, parallelism, and computational speed compared with conventional deep neural networks (DNNs), which have intrinsic limitations when implemented on digital platforms. However, inversely mapping algorithm-trained physical model parameters onto real-world optical devices with discrete values is a non-trivial task as existing optical devices have non-unified discrete levels and non-monotonic properties. This work proposes a novel device-to-system hardware-software codesign framework, which enables efficient physics-aware training of DONNs w.r.t arbitrary experimental measured optical devices across layers. Specifically, Gumbel-Softmax is employed to enable differentiable discrete mapping from real-world device parameters into the forward function of DONNs, where the physical parameters in DONNs can be trained by simply minimizing the loss function of the ML task. The results have demonstrated that our proposed framework offers significant advantages over conventional quantization-based methods, especially with low-precision optical devices. Finally, the proposed algorithm is fully verified with physical experimental optical systems in low-precision settings.