# All-Optical Segmentation via Diffractive Neural Networks for Autonomous Driving

arXiv: https://arxiv.org/abs/2602.07717 · HTML: https://mlsanity.com/paper/2602.07717/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 26 of its 40 references that are in the map.
- First posted on arXiv 7 Feb 2026.

## Builds on (most cited first, 26 of 26)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [Fully Convolutional Networks for Semantic Segmentation](https://mlsanity.com/paper/1411.4038/) (2014)
- [The Cityscapes Dataset for Semantic Urban Scene Understanding](https://mlsanity.com/paper/1604.01685/) (2016)
- [MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications](https://mlsanity.com/paper/1704.04861/) (2017)
- [V-Net: Fully Convolutional Neural Networks for Volumetric Medical Image Segmentation](https://mlsanity.com/paper/1606.04797/) (2016)
- [Planning-oriented Autonomous Driving](https://mlsanity.com/paper/2212.10156/) (2022)
- [YOLOv9: Learning What You Want to Learn Using Programmable Gradient Information](https://mlsanity.com/paper/2402.13616/) (2024)
- [A Survey of Autonomous Driving: Common Practices and Emerging Technologies](https://mlsanity.com/paper/1906.05113/) (2019)
- [Learning Deconvolution Network for Semantic Segmentation](https://mlsanity.com/paper/1505.04366/) (2015)
- [All-Optical Machine Learning Using Diffractive Deep Neural Networks](https://mlsanity.com/paper/1804.08711/) (2018)
- [Large-scale neuromorphic optoelectronic computing with a reconfigurable diffractive processing unit](https://mlsanity.com/paper/2008.11659/) (2020)
- [SceneDiffuser: Efficient and Controllable Driving Simulation Initialization and Rollout](https://mlsanity.com/paper/2412.12129/) (2024)
- [Training of photonic neural networks through in situ backpropagation](https://mlsanity.com/paper/1805.09943/) (2018)
- [RT-DETRv3: Real-time End-to-End Object Detection with Hierarchical Dense Positive Supervision](https://mlsanity.com/paper/2409.08475/) (2024)
- [Analysis of Diffractive Optical Neural Networks and Their Integration with Electronic Neural Networks](https://mlsanity.com/paper/1810.01916/) (2018)
- [Metasurface-Enabled On-Chip Multiplexed Diffractive Neural Networks in the Visible](https://mlsanity.com/paper/2107.07873/) (2021)
- [End-to-End Deep Learning of Lane Detection and Path Prediction for Real-Time Autonomous Driving](https://mlsanity.com/paper/2102.04738/) (2021)
- [Scale-, shift- and rotation-invariant diffractive optical networks](https://mlsanity.com/paper/2010.12747/) (2020)
- [All-optical graph representation learning using integrated diffractive photonic computing units](https://mlsanity.com/paper/2204.10978/) (2022)
- [Understanding Time Variations of DNN Inference in Autonomous Driving](https://mlsanity.com/paper/2209.05487/) (2022)
- [Real-time Multi-Task Diffractive Deep Neural Networks via Hardware-Software Co-design](https://mlsanity.com/paper/2012.08906/) (2020)

## Abstract (arXiv)

Semantic segmentation and lane detection are crucial tasks in autonomous driving systems. Conventional approaches predominantly rely on deep neural networks (DNNs), which incur high energy costs due to extensive analog-to-digital conversions and large-scale image computations required for low-latency, real-time responses. Diffractive optical neural networks (DONNs) have shown promising advantages over conventional DNNs on digital or optoelectronic computing platforms in energy efficiency. By performing all-optical image processing via light diffraction at the speed of light, DONNs save computation energy costs while reducing the overhead associated with analog-to-digital conversions by all-optical encoding and computing. In this work, we propose a novel all-optical computing framework for RGB image segmentation and lane detection in autonomous driving applications. Our experimental results demonstrate the effectiveness of the DONN system for image segmentation on the CityScapes dataset. Additionally, we conduct case studies on lane detection using a customized indoor track dataset and simulated driving scenarios in CARLA, where we further evaluate the model's generalizability under diverse environmental conditions.