# Neuromorphic Optical Tracking and Imaging of Randomly Moving Targets through Strongly Scattering Media

arXiv: https://arxiv.org/abs/2501.03874 · HTML: https://mlsanity.com/paper/2501.03874/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 23 of its 82 references that are in the map.
- First posted on arXiv 7 Jan 2025.

## Builds on (most cited first, 23 of 23)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Event-based Vision: A Survey](https://mlsanity.com/paper/1904.08405/) (2019)
- [Surrogate Gradient Learning in Spiking Neural Networks](https://mlsanity.com/paper/1901.09948/) (2019)
- [Holistically-Nested Edge Detection](https://mlsanity.com/paper/1504.06375/) (2015)
- [Converting Static Image Datasets to Spiking Neuromorphic Datasets Using Saccades](https://mlsanity.com/paper/1507.07629/) (2015)
- [Training Spiking Neural Networks Using Lessons From Deep Learning](https://mlsanity.com/paper/2109.12894/) (2021)
- [Spike-driven Transformer V2: Meta Spiking Neural Network Architecture Inspiring the Design of Next-generation Neuromorphic Chips](https://mlsanity.com/paper/2404.03663/) (2024)
- [Efficient Neuromorphic Signal Processing with Loihi 2](https://mlsanity.com/paper/2111.03746/) (2021)
- [DIET-SNN: Direct Input Encoding With Leakage and Threshold Optimization in Deep Spiking Neural Networks](https://mlsanity.com/paper/2008.03658/) (2020)
- [HIRE-SNN: Harnessing the Inherent Robustness of Energy-Efficient Deep Spiking Neural Networks by Training with Crafted Input Noise](https://mlsanity.com/paper/2110.11417/) (2021)
- [SpikingResformer: Bridging ResNet and Vision Transformer in Spiking Neural Networks](https://mlsanity.com/paper/2403.14302/) (2024)
- [Exploiting Neuron and Synapse Filter Dynamics in Spatial Temporal Learning of Deep Spiking Neural Network](https://mlsanity.com/paper/2003.02944/) (2020)
- [Optical flow estimation from event-based cameras and spiking neural networks](https://mlsanity.com/paper/2302.06492/) (2023)
- [A Fully Spiking Hybrid Neural Network for Energy-Efficient Object Detection](https://mlsanity.com/paper/2104.10719/) (2021)
- [Training Full Spike Neural Networks via Auxiliary Accumulation Pathway](https://mlsanity.com/paper/2301.11929/) (2023)
- [Spiking Neural Networks for Frame-based and Event-based Single Object Localization](https://mlsanity.com/paper/2206.06506/) (2022)
- [EventLFM: Event Camera integrated Fourier Light Field Microscopy for Ultrafast 3D imaging](https://mlsanity.com/paper/2310.00730/) (2023)
- [High Resolution, Deep Imaging Using Confocal Time-of-flight Diffuse Optical Tomography](https://mlsanity.com/paper/2101.11680/) (2021)
- [An Asynchronous Wireless Network for Capturing Event-Driven Data from Large Populations of Autonomous Sensors](https://mlsanity.com/paper/2305.12293/) (2023)
- [Tracking moving objects through scattering media via speckle correlations](https://mlsanity.com/paper/2202.10804/) (2022)
- [Enhancing Speckle Statistics for Imaging inside Scattering Media](https://mlsanity.com/paper/2203.14214/) (2022)
- [A Spiking Neural Network Decoder for Implantable Brain Machine Interfaces and its Sparsity-aware Deployment on RISC-V Microcontrollers](https://mlsanity.com/paper/2405.02146/) (2024)
- [Noninvasive super-resolution imaging through scattering media](https://mlsanity.com/paper/1906.03823/) (2019)

## Abstract (arXiv)

Tracking and acquiring simultaneous optical images of randomly moving targets obscured by scattering media remains a challenging problem of importance to many applications that require precise object localization and identification. In this work we develop an end-to-end neuromorphic optical engineering and computational approach to demonstrate how to track and image normally invisible objects by combining an event detecting camera with a multistage neuromorphic deep learning strategy. Photons emerging from dense scattering media are detected by the event camera and converted to pixel-wise asynchronized spike trains - a first step in isolating object-specific information from the dominant uninformative background. Spiking data is fed into a deep spiking neural network (SNN) engine where object tracking and image reconstruction are performed by two separate yet interconnected modules running in parallel in discrete time steps over the event duration. Through benchtop experiments we demonstrate tracking and imaging randomly moving objects in dense turbid media as well as image reconstruction of spatially stationary but optically dynamic objects. Standardized character sets serve as representative proxies for geometrically complex objects, underscoring the method's generality. The results highlight the advantages of a fully neuromorphic approach in meeting a major imaging technology with high computational efficiency and low power consumption.