# Transferable polychromatic optical encoder for neural networks

arXiv: https://arxiv.org/abs/2411.02697 · HTML: https://mlsanity.com/paper/2411.02697/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 63 references that are in the map.
- First posted on arXiv 5 Nov 2024.

## Builds on (most cited first, 21 of 21)

- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Paying More Attention to Attention: Improving the Performance of Convolutional Neural Networks via Attention Transfer](https://mlsanity.com/paper/1612.03928/) (2016)
- [Relational Knowledge Distillation](https://mlsanity.com/paper/1904.05068/) (2019)
- [Sequence-Level Knowledge Distillation](https://mlsanity.com/paper/1606.07947/) (2016)
- [Contrastive Representation Distillation](https://mlsanity.com/paper/1910.10699/) (2019)
- [Pruning and Quantization for Deep Neural Network Acceleration: A Survey](https://mlsanity.com/paper/2101.09671/) (2021)
- [All-Optical Machine Learning Using Diffractive Deep Neural Networks](https://mlsanity.com/paper/1804.08711/) (2018)
- [The physics of optical computing](https://mlsanity.com/paper/2308.00088/) (2023)
- [Image sensing with multilayer, nonlinear optical neural networks](https://mlsanity.com/paper/2207.14293/) (2022)
- [Nonlinear optical encoding enabled by recurrent linear scattering](https://mlsanity.com/paper/2307.08558/) (2023)
- [Single-Shot Optical Neural Network](https://mlsanity.com/paper/2205.09103/) (2022)
- [Spatially Varying Nanophotonic Neural Networks](https://mlsanity.com/paper/2308.03407/) (2023)
- [Optical Transformers](https://mlsanity.com/paper/2302.10360/) (2023)
- [Photonic Advantage of Optical Encoders](https://mlsanity.com/paper/2305.01743/) (2023)
- [Knowledge Distillation Circumvents Nonlinearity for Optical Convolutional Neural Networks](https://mlsanity.com/paper/2102.13323/) (2021)
- [Compressed Meta-Optical Encoder for Image Classification](https://mlsanity.com/paper/2406.06534/) (2024)
- [All-optical image classification through unknown random diffusers using a single-pixel diffractive network](https://mlsanity.com/paper/2208.03968/) (2022)
- [KL-Divergence-Based Region Proposal Network for Object Detection](https://mlsanity.com/paper/2005.11220/) (2020)
- [BI-MAML: Balanced Incremental Approach for Meta Learning](https://mlsanity.com/paper/2006.07412/) (2020)
- [Time-lapse image classification using a diffractive neural network](https://mlsanity.com/paper/2208.10802/) (2022)
- [Using Scalable Computer Vision to Automate High-throughput Semiconductor Characterization](https://mlsanity.com/paper/2304.14408/) (2023)

## Built on by (most cited first, 3 of 3)

- [Neural Tangent Knowledge Distillation for Optical Convolutional Networks](https://mlsanity.com/paper/2508.08421/) (2025)
- [Spatially Parallel All-optical Neural Networks](https://mlsanity.com/paper/2509.23611/) (2025)
- [ReDON: Recurrent Diffractive Optical Neural Processor with Reconfigurable Self-Modulated Nonlinearity](https://mlsanity.com/paper/2602.23616/) (2026)

## Abstract (arXiv)

Artificial neural networks (ANNs) have fundamentally transformed the field of computer vision, providing unprecedented performance. However, these ANNs for image processing demand substantial computational resources, often hindering real-time operation. In this paper, we demonstrate an optical encoder that can perform convolution simultaneously in three color channels during the image capture, effectively implementing several initial convolutional layers of a ANN. Such an optical encoding results in ~24,000 times reduction in computational operations, with a state-of-the art classification accuracy (~73.2%) in free-space optical system. In addition, our analog optical encoder, trained for CIFAR-10 data, can be transferred to the ImageNet subset, High-10, without any modifications, and still exhibits moderate accuracy. Our results evidence the potential of hybrid optical/digital computer vision system in which the optical frontend can pre-process an ambient scene to reduce the energy and latency of the whole computer vision system.