# Adaptive Illumination Control for Robot Perception

arXiv: https://arxiv.org/abs/2602.15900 · HTML: https://mlsanity.com/paper/2602.15900/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 17 of its 45 references that are in the map.
- First posted on arXiv 13 Feb 2026.

## Builds on (most cited first, 17 of 17)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Optuna: A Next-generation Hyperparameter Optimization Framework](https://mlsanity.com/paper/1907.10902/) (2019)
- [Grounding Image Matching in 3D with MASt3R](https://mlsanity.com/paper/2406.09756/) (2024)
- [ORB-SLAM3: An Accurate Open-Source Library for Visual, Visual-Inertial and Multi-Map SLAM](https://mlsanity.com/paper/2007.11898/) (2020)
- [Direct Sparse Odometry](https://mlsanity.com/paper/1607.02565/) (2016)
- [Interactive Perception: Leveraging Action in Perception and Perception in Action](https://mlsanity.com/paper/1604.03670/) (2016)
- [A Dataset of Multi-Illumination Images in the Wild](https://mlsanity.com/paper/1910.08131/) (2019)
- [LumiNet: Latent Intrinsics Meets Diffusion Models for Indoor Scene Relighting](https://mlsanity.com/paper/2412.00177/) (2024)
- [Direct Intrinsics: Learning Albedo-Shading Decomposition by Convolutional Regression](https://mlsanity.com/paper/1512.02311/) (2015)
- [Latent Intrinsics Emerge from Training to Relight](https://mlsanity.com/paper/2405.21074/) (2024)
- [Learned Camera Gain and Exposure Control for Improved Visual Feature Detection and Matching](https://mlsanity.com/paper/2102.04341/) (2021)
- [Learning to Control Camera Exposure via Reinforcement Learning](https://mlsanity.com/paper/2404.01636/) (2024)
- [Learning-based Image Enhancement for Visual Odometry in Challenging HDR Environments](https://mlsanity.com/paper/1707.01274/) (2017)
- [Computational Flash Photography through Intrinsics](https://mlsanity.com/paper/2306.06089/) (2023)
- [Efficient Camera Exposure Control for Visual Odometry via Deep Reinforcement Learning](https://mlsanity.com/paper/2408.17005/) (2024)
- [Active Illumination Control in Low-Light Environments using NightHawk](https://mlsanity.com/paper/2506.06394/) (2025)
- [Reproducible Evaluation of Camera Auto-Exposure Methods in the Field: Platform, Benchmark and Lessons Learned](https://mlsanity.com/paper/2506.18844/) (2025)

## Abstract (arXiv)

Robot perception under low light or high dynamic range is usually improved downstream - via more robust feature extraction, image enhancement, or closed-loop exposure control. However, all of these approaches are limited by the image captured these conditions. An alternate approach is to utilize a programmable onboard light that adds to ambient illumination and improves captured images. However, it is not straightforward to predict its impact on image formation. Illumination interacts nonlinearly with depth, surface reflectance, and scene geometry. It can both reveal structure and induce failure modes such as specular highlights and saturation. We introduce Lightning, a closed-loop illumination-control framework for visual SLAM that combines relighting, offline optimization, and imitation learning. This is performed in three stages. First, we train a Co-Located Illumination Decomposition (CLID) relighting model that decomposes a robot observation into an ambient component and a light-contribution field. CLID enables physically consistent synthesis of the same scene under alternative light intensities and thereby creates dense multi-intensity training data without requiring us to repeatedly re-run trajectories. Second, using these synthesized candidates, we formulate an offline Optimal Intensity Schedule (OIS) problem that selects illumination levels over a sequence trading off SLAM-relevant image utility against power consumption and temporal smoothness. Third, we distill this ideal solution into a real-time controller through behavior cloning, producing an Illumination Control Policy (ILC) that generalizes beyond the initial training distribution and runs online on a mobile robot to command discrete light-intensity levels. Across our evaluation, Lightning substantially improves SLAM trajectory robustness while reducing unnecessary illumination power.