# Compute-first optical detection for noise-resilient visual perception

arXiv: https://arxiv.org/abs/2403.09612 · HTML: https://mlsanity.com/paper/2403.09612/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 12 of its 54 references that are in the map.
- First posted on arXiv 14 Mar 2024.

## Builds on (most cited first, 12 of 12)

- [YOLOv3: An Incremental Improvement](https://mlsanity.com/paper/1804.02767/) (2018)
- [Learning both Weights and Connections for Efficient Neural Networks](https://mlsanity.com/paper/1506.02626/) (2015)
- [LLVIP: A Visible-infrared Paired Dataset for Low-light Vision](https://mlsanity.com/paper/2108.10831/) (2021)
- [EMNIST: an extension of MNIST to handwritten letters](https://mlsanity.com/paper/1702.05373/) (2017)
- [All-Optical Machine Learning Using Diffractive Deep Neural Networks](https://mlsanity.com/paper/1804.08711/) (2018)
- [Misalignment Resilient Diffractive Optical Networks](https://mlsanity.com/paper/2005.11464/) (2020)
- [Image sensing with multilayer, nonlinear optical neural networks](https://mlsanity.com/paper/2207.14293/) (2022)
- [Universal Linear Intensity Transformations Using Spatially-Incoherent Diffractive Processors](https://mlsanity.com/paper/2303.13037/) (2023)
- [Single-Shot Optical Neural Network](https://mlsanity.com/paper/2205.09103/) (2022)
- [Dual adaptive training of photonic neural networks](https://mlsanity.com/paper/2212.06141/) (2022)
- [All-optical image denoising using a diffractive visual processor](https://mlsanity.com/paper/2309.09215/) (2023)
- [A photosensor employing data-driven binning for ultrafast image recognition](https://mlsanity.com/paper/2111.10612/) (2021)

## Abstract (arXiv)

In the context of visual perception, the optical signal from a scene is transferred into the electronic domain by detectors in the form of image data, which are then processed for the extraction of visual information. In noisy and weak-signal environments such as thermal imaging for night vision applications, however, the performance of neural computing tasks faces a significant bottleneck due to the inherent degradation of data quality upon noisy detection. Here, we propose a concept of optical signal processing before detection to address this issue. We demonstrate that spatially redistributing optical signals through a properly designed linear transformer can enhance the detection noise resilience of visual perception tasks, as benchmarked with the MNIST classification. Our idea is supported by a quantitative analysis detailing the relationship between signal concentration and noise robustness, as well as its practical implementation in an incoherent imaging system. This compute-first detection scheme can pave the way for advancing infrared machine vision technologies widely used for industrial and defense applications.