# An Optical Frontend for a Convolutional Neural Network

arXiv: https://arxiv.org/abs/1901.03661 · HTML: https://mlsanity.com/paper/1901.03661/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 23 Dec 2018.

## Built on by (most cited first, 18 of 18)

- [Image sensing with multilayer, nonlinear optical neural networks](https://mlsanity.com/paper/2207.14293/) (2022)
- [Spatially Varying Nanophotonic Neural Networks](https://mlsanity.com/paper/2308.03407/) (2023)
- [Photonic Advantage of Optical Encoders](https://mlsanity.com/paper/2305.01743/) (2023)
- [Knowledge Distillation Circumvents Nonlinearity for Optical Convolutional Neural Networks](https://mlsanity.com/paper/2102.13323/) (2021)
- [Compressed Meta-Optical Encoder for Image Classification](https://mlsanity.com/paper/2406.06534/) (2024)
- [Classification and reconstruction of spatially overlapping phase images using diffractive optical networks](https://mlsanity.com/paper/2108.07977/) (2021)
- [FatNet: High Resolution Kernels for Classification Using Fully Convolutional Optical Neural Networks](https://mlsanity.com/paper/2210.16914/) (2022)
- [Neural Tangent Knowledge Distillation for Optical Convolutional Networks](https://mlsanity.com/paper/2508.08421/) (2025)
- [Differentiable Microscopy Designs an All Optical Phase Retrieval Microscope](https://mlsanity.com/paper/2203.14944/) (2022)
- [SP2RINT: Spatially-Decoupled Physics-Inspired Progressive Inverse Optimization for Scalable, PDE-Constrained Meta-Optical Neural Network Training](https://mlsanity.com/paper/2505.18377/) (2025)
- [Intelligent Multi-channel Meta-imagers for Accelerating Machine Vision](https://mlsanity.com/paper/2306.07365/) (2023)
- [Metasurface-generated large and arbitrary analog convolution kernels for accelerated machine vision](https://mlsanity.com/paper/2409.18614/) (2024)
- [Computational metaoptics for imaging](https://mlsanity.com/paper/2411.09133/) (2024)
- [ConvShareViT: Enhancing Vision Transformers with Convolutional Attention Mechanisms for Free-Space Optical Accelerators](https://mlsanity.com/paper/2504.11517/) (2025)
- [Programmable metasurfaces for future photonic artificial intelligence](https://mlsanity.com/paper/2505.11659/) (2025)
- [De-occluding broadband metalens](https://mlsanity.com/paper/2601.19403/) (2026)
- [End-to-End Optimization of Incoherent Imaging for Classification Under Detector-Limited Readout](https://mlsanity.com/paper/2606.09792/) (2026)
- [VLM-Aware Meta-Optic Front-End Design for Frozen Vision-Language Models](https://mlsanity.com/paper/2606.27646/) (2026)

## Abstract (arXiv)

The parallelism of optics and the miniaturization of optical components using nanophotonic structures, such as metasurfaces present a compelling alternative to electronic implementations of convolutional neural networks. The lack of a low-power optical nonlinearity, however, requires slow and energy-inefficient conversions between the electronic and optical domains. Here, we design an architecture which utilizes a single electrical to optical conversion by designing a free-space optical frontend unit that implements the linear operations of the first layer with the subsequent layers realized electronically. Speed and power analysis of the architecture indicates that the hybrid photonic-electronic architecture outperforms sole electronic architecture for large image sizes and kernels. Benchmarking of the photonic-electronic architecture on a modified version of AlexNet achieves a classification accuracy of 87% on images from the Kaggle Cats and Dogs challenge database.