# DarkGS: Learning Neural Illumination and 3D Gaussians Relighting for Robotic Exploration in the Dark

arXiv: https://arxiv.org/abs/2403.10814 · HTML: https://mlsanity.com/paper/2403.10814/

## Key facts

- Cited by 25 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 42 references that are in the map.
- First posted on arXiv 16 Mar 2024.

## Builds on (most cited first, 20 of 20)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [3D Gaussian Splatting for Real-Time Radiance Field Rendering](https://mlsanity.com/paper/2308.04079/) (2023)
- [Instant Neural Graphics Primitives with a Multiresolution Hash Encoding](https://mlsanity.com/paper/2201.05989/) (2022)
- [CARLA: An Open Urban Driving Simulator](https://mlsanity.com/paper/1711.03938/) (2017)
- [Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fields](https://mlsanity.com/paper/2103.13415/) (2021)
- [ORB-SLAM: a Versatile and Accurate Monocular SLAM System](https://mlsanity.com/paper/1502.00956/) (2015)
- [ORB-SLAM3: An Accurate Open-Source Library for Visual, Visual-Inertial and Multi-Map SLAM](https://mlsanity.com/paper/2007.11898/) (2020)
- [NeRV: Neural Reflectance and Visibility Fields for Relighting and View Synthesis](https://mlsanity.com/paper/2012.03927/) (2020)
- [NeRF in the Dark: High Dynamic Range View Synthesis from Noisy Raw Images](https://mlsanity.com/paper/2111.13679/) (2021)
- [Text-to-3D using Gaussian Splatting](https://mlsanity.com/paper/2309.16585/) (2023)
- [A Survey on 3D Gaussian Splatting](https://mlsanity.com/paper/2401.03890/) (2024)
- [Neural Reflectance Fields for Appearance Acquisition](https://mlsanity.com/paper/2008.03824/) (2020)
- [Tangent Space Backpropagation for 3D Transformation Groups](https://mlsanity.com/paper/2103.12032/) (2021)
- [NeBula: Quest for Robotic Autonomy in Challenging Environments; TEAM CoSTAR at the DARPA Subterranean Challenge](https://mlsanity.com/paper/2103.11470/) (2021)
- [Automatic Extrinsic Calibration Method for LiDAR and Camera Sensor Setups](https://mlsanity.com/paper/2101.04431/) (2021)
- [Beyond NeRF Underwater: Learning Neural Reflectance Fields for True Color Correction of Marine Imagery](https://mlsanity.com/paper/2304.03384/) (2023)
- [Gaussian Shadow Casting for Neural Characters](https://mlsanity.com/paper/2401.06116/) (2024)
- [LiRaFusion: Deep Adaptive LiDAR-Radar Fusion for 3D Object Detection](https://mlsanity.com/paper/2402.11735/) (2024)
- [Single-image RGB Photometric Stereo With Spatially-varying Albedo](https://mlsanity.com/paper/1609.04079/) (2016)
- [Light Pose Calibration for Camera-light Vision Systems](https://mlsanity.com/paper/2006.15389/) (2020)

## Built on by (most cited first, 25 of 25)

- [Lighting Every Darkness with 3DGS: Fast Training and Real-Time Rendering for HDR View Synthesis](https://mlsanity.com/paper/2406.06216/) (2024)
- [Luminance-GS: Adapting 3D Gaussian Splatting to Challenging Lighting Conditions with View-Adaptive Curve Adjustment](https://mlsanity.com/paper/2504.01503/) (2025)
- [Gaussian in the Dark: Real-Time View Synthesis From Inconsistent Dark Images Using Gaussian Splatting](https://mlsanity.com/paper/2408.09130/) (2024)
- [RecGS: Removing Water Caustic with Recurrent Gaussian Splatting](https://mlsanity.com/paper/2407.10318/) (2024)
- [SplaTraj: Camera Trajectory Generation with Semantic Gaussian Splatting](https://mlsanity.com/paper/2410.06014/) (2024)
- [Unifying Color and Lightness Correction with View-Adaptive Curve Adjustment for Robust 3D Novel View Synthesis](https://mlsanity.com/paper/2602.18322/) (2026)
- [LLGS: Unsupervised Gaussian Splatting for Image Enhancement and Reconstruction in Pure Dark Environment](https://mlsanity.com/paper/2503.18640/) (2025)
- [PhotoReg: Photometrically Registering 3D Gaussian Splatting Models](https://mlsanity.com/paper/2410.05044/) (2024)
- [Dark-EvGS: Event Camera as an Eye for Radiance Field in the Dark](https://mlsanity.com/paper/2507.11931/) (2025)
- [DOSE3 : Diffusion-based Out-of-distribution detection on SE(3) trajectories](https://mlsanity.com/paper/2502.16725/) (2025)
- [3D Foundation Models Enable Simultaneous Geometry and Pose Estimation of Grasped Objects](https://mlsanity.com/paper/2407.10331/) (2024)
- [Infinite Leagues Under the Sea: Photorealistic 3D Underwater Terrain Generation by Latent Fractal Diffusion Models](https://mlsanity.com/paper/2503.06784/) (2025)
- [GS-I3: Gaussian Splatting for Surface Reconstruction from Illumination-Inconsistent Images](https://mlsanity.com/paper/2503.12335/) (2025)
- [GraphSeg: Segmented 3D Representations via Graph Edge Addition and Contraction](https://mlsanity.com/paper/2504.03129/) (2025)
- [Relative Illumination Fields: Learning Medium and Light Independent Underwater Scenes](https://mlsanity.com/paper/2504.10024/) (2025)
- [Bi-Manual Joint Camera Calibration and Scene Representation](https://mlsanity.com/paper/2505.24819/) (2025)
- [Endo-4DGX: Robust Endoscopic Scene Reconstruction and Illumination Correction with Gaussian Splatting](https://mlsanity.com/paper/2506.23308/) (2025)
- [Optimizing 3D Gaussian Splattering for Mobile GPUs](https://mlsanity.com/paper/2511.16298/) (2025)
- [Naka-GS: Naka-Guided Chroma Correction and Point-Prior Compaction for Low-Light 3D Gaussian Splatting](https://mlsanity.com/paper/2604.11142/) (2026)
- [Unifying Scene Representation and Hand-Eye Calibration with 3D Foundation Models](https://mlsanity.com/paper/2404.11683/) (2024)
- [SplatBright: Generalizable Low-Light Scene Reconstruction from Sparse Views via Physically-Guided Gaussian Enhancement](https://mlsanity.com/paper/2512.18655/) (2025)
- [Nighttime Autonomous Driving Scene Reconstruction with Physically-Based Gaussian Splatting](https://mlsanity.com/paper/2602.13549/) (2026)
- [Scene-Adaptive Nonlinear Tone Curves for Pseudo Ground-Truth Generation in Low-Light 3D Gaussian Splatting](https://mlsanity.com/paper/2606.11841/) (2026)
- [SpotlessGS: Relightable 3D Gaussian Splatting under Dynamic Illumination for Robotic Perception](https://mlsanity.com/paper/2608.14713/) (2026)
- [EvenSplat: Coupled 2D-3D Decomposition for Gaussian Splatting under Exposure and Illumination Variation](https://mlsanity.com/paper/2610.01876/) (2026)

## Abstract (arXiv)

Humans have the remarkable ability to construct consistent mental models of an environment, even under limited or varying levels of illumination. We wish to endow robots with this same capability. In this paper, we tackle the challenge of constructing a photorealistic scene representation under poorly illuminated conditions and with a moving light source. We approach the task of modeling illumination as a learning problem, and utilize the developed illumination model to aid in scene reconstruction. We introduce an innovative framework that uses a data-driven approach, Neural Light Simulators (NeLiS), to model and calibrate the camera-light system. Furthermore, we present DarkGS, a method that applies NeLiS to create a relightable 3D Gaussian scene model capable of real-time, photorealistic rendering from novel viewpoints. We show the applicability and robustness of our proposed simulator and system in a variety of real-world environments.