# Event Spectroscopy: Event-based Multispectral and Depth Sensing using Structured Light

arXiv: https://arxiv.org/abs/2509.06741 · HTML: https://mlsanity.com/paper/2509.06741/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 53 references that are in the map.
- First posted on arXiv 8 Sep 2025.

## Builds on (most cited first, 17 of 17)

- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [Event-based Vision: A Survey](https://mlsanity.com/paper/1904.08405/) (2019)
- [Learning High-Speed Flight in the Wild](https://mlsanity.com/paper/2110.05113/) (2021)
- [EV-SegNet: Semantic Segmentation for Event-based Cameras](https://mlsanity.com/paper/1811.12039/) (2018)
- [Single-shot Hyperspectral-Depth Imaging with Learned Diffractive Optics](https://mlsanity.com/paper/2009.00463/) (2020)
- [CED: Color Event Camera Dataset](https://mlsanity.com/paper/1904.10772/) (2019)
- [An Autonomous Drone for Search and Rescue in Forests using Airborne Optical Sectioning](https://mlsanity.com/paper/2105.04328/) (2021)
- [Shining light on the DVS pixel: A tutorial and discussion about biasing and optimization](https://mlsanity.com/paper/2304.04706/) (2023)
- [ESL: Event-based Structured Light](https://mlsanity.com/paper/2111.15510/) (2021)
- [Event Guided Depth Sensing](https://mlsanity.com/paper/2110.10505/) (2021)
- [Large-scale Autonomous Flight with Real-time Semantic SLAM under Dense Forest Canopy](https://mlsanity.com/paper/2109.06479/) (2021)
- [Time-Efficient Light-Field Acquisition Using Coded Aperture and Events](https://mlsanity.com/paper/2403.07244/) (2024)
- [Noise2Image: Noise-Enabled Static Scene Recovery for Event Cameras](https://mlsanity.com/paper/2404.01298/) (2024)
- [CodedEvents: Optimal Point-Spread-Function Engineering for 3D-Tracking with Event Cameras](https://mlsanity.com/paper/2406.09409/) (2024)
- [Seeing like a Cephalopod: Colour Vision with a Monochrome Event Camera](https://mlsanity.com/paper/2504.10984/) (2025)
- [Event-based RGB-D sensing with structured light](https://mlsanity.com/paper/2207.11605/) (2022)

## Built on by (most cited first, 3 of 3)

- [Learning to Remove Lens Flare in Event Camera](https://mlsanity.com/paper/2512.09016/) (2025)
- [E-RGB-D: Real-Time Event-Based Perception with Structured Light](https://mlsanity.com/paper/2512.18429/) (2025)
- [EventEgoHands++: Event-based Egocentric 3D Hand Mesh Reconstruction with Real Dataset](https://mlsanity.com/paper/2609.17189/) (2026)

## Abstract (arXiv)

Uncrewed aerial vehicles (UAVs) are increasingly deployed in forest environments for tasks such as environmental monitoring and search and rescue, which require safe navigation through dense foliage and precise data collection. Traditional sensing approaches, including passive multispectral and RGB imaging, suffer from latency, poor depth resolution, and strong dependence on ambient light - especially under forest canopies. In this work, we present a novel event spectroscopy system that simultaneously enables high-resolution, low-latency depth reconstruction with integrated multispectral imaging using a single sensor. Depth is reconstructed using structured light, and by modulating the wavelength of the projected structured light, our system captures spectral information in controlled bands between 650 nm and 850 nm. We demonstrate up to 60% improvement in RMSE over commercial depth sensors and validate the spectral accuracy against a reference spectrometer and commercial multispectral cameras, demonstrating comparable performance. A portable version limited to RGB (3 wavelengths) is used to collect real-world depth and spectral data from a Masoala Rainforest. We demonstrate the use of this prototype for color image reconstruction and material differentiation between leaves and branches using spectral and depth data. Our results show that adding depth (available at no extra effort with our setup) to material differentiation improves the accuracy by over 30% compared to color-only method. Our system, tested in both lab and real-world rainforest environments, shows strong performance in depth estimation, RGB reconstruction, and material differentiation - paving the way for lightweight, integrated, and robust UAV perception and data collection in complex natural environments.