# Light the Night: A Multi-Condition Diffusion Framework for Unpaired Low-Light Enhancement in Autonomous Driving

arXiv: https://arxiv.org/abs/2404.04804 · HTML: https://mlsanity.com/paper/2404.04804/

## Key facts

- Cited by 35 papers in the map (data through 14 Oct 2026).
- Placed in VLMs for autonomous driving, one of 9 directions in End-to-end autonomous driving.
- Builds on 41 of its 58 references that are in the map.
- First posted on arXiv 7 Apr 2024.

## Placed in

- End-to-end autonomous driving / [VLMs for autonomous driving](https://mlsanity.com/driving/vlms-for-autonomous-driving/)

## Builds on (most cited first, 30 of 41)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Training language models to follow instructions with human feedback](https://mlsanity.com/paper/2203.02155/) (2022)
- [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022)
- [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022)
- [Adding Conditional Control to Text-to-Image Diffusion Models](https://mlsanity.com/paper/2302.05543/) (2023)
- [nuScenes: A multimodal dataset for autonomous driving](https://mlsanity.com/paper/1903.11027/) (2019)
- [Flamingo: a Visual Language Model for Few-Shot Learning](https://mlsanity.com/paper/2204.14198/) (2022)
- [BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation](https://mlsanity.com/paper/2201.12086/) (2022)
- [DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation](https://mlsanity.com/paper/2208.12242/) (2022)
- [Align before Fuse: Vision and Language Representation Learning with Momentum Distillation](https://mlsanity.com/paper/2107.07651/) (2021)
- [Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfer](https://mlsanity.com/paper/1907.01341/) (2019)
- [MUSIQ: Multi-scale Image Quality Transformer](https://mlsanity.com/paper/2108.05997/) (2021)
- [Vector Quantized Diffusion Model for Text-to-Image Synthesis](https://mlsanity.com/paper/2111.14822/) (2021)
- [DiffusionCLIP: Text-Guided Diffusion Models for Robust Image Manipulation](https://mlsanity.com/paper/2110.02711/) (2021)
- [BEVDepth: Acquisition of Reliable Depth for Multi-view 3D Object Detection](https://mlsanity.com/paper/2206.10092/) (2022)
- [Zero-Reference Deep Curve Estimation for Low-Light Image Enhancement](https://mlsanity.com/paper/2001.06826/) (2020)
- [MANIQA: Multi-dimension Attention Network for No-Reference Image Quality Assessment](https://mlsanity.com/paper/2204.08958/) (2022)
- [EnlightenGAN: Deep Light Enhancement without Paired Supervision](https://mlsanity.com/paper/1906.06972/) (2019)
- [NIMA: Neural Image Assessment](https://mlsanity.com/paper/1709.05424/) (2017)
- [Retinex-inspired Unrolling with Cooperative Prior Architecture Search for Low-light Image Enhancement](https://mlsanity.com/paper/2012.05609/) (2020)
- [Blind Image Quality Assessment via Vision-Language Correspondence: A Multitask Learning Perspective](https://mlsanity.com/paper/2303.14968/) (2023)
- [An Empirical Study of Training End-to-End Vision-and-Language Transformers](https://mlsanity.com/paper/2111.02387/) (2021)
- [Unprocessing Images for Learned Raw Denoising](https://mlsanity.com/paper/1811.11127/) (2018)
- [Generative Diffusion Prior for Unified Image Restoration and Enhancement](https://mlsanity.com/paper/2304.01247/) (2023)
- [Learning to Enhance Low-Light Image via Zero-Reference Deep Curve Estimation](https://mlsanity.com/paper/2103.00860/) (2021)
- [No-Reference Image Quality Assessment via Transformers, Relative Ranking, and Self-Consistency](https://mlsanity.com/paper/2108.06858/) (2021)
- [Diff-Retinex: Rethinking Low-light Image Enhancement with A Generative Diffusion Model](https://mlsanity.com/paper/2308.13164/) (2023)
- [Iterative Prompt Learning for Unsupervised Backlit Image Enhancement](https://mlsanity.com/paper/2303.17569/) (2023)

## Built on by (most cited first, 30 of 35)

- [OpenEMMA: Open-Source Multimodal Model for End-to-End Autonomous Driving](https://mlsanity.com/paper/2412.15208/) (2024)
- [SleeperMark: Towards Robust Watermark against Fine-Tuning Text-to-image Diffusion Models](https://mlsanity.com/paper/2412.04852/) (2024)
- [UniUIR: Considering Underwater Image Restoration as An All-in-One Learner](https://mlsanity.com/paper/2501.12981/) (2025)
- [Edit Away and My Face Will not Stay: Personal Biometric Defense against Malicious Generative Editing](https://mlsanity.com/paper/2411.16832/) (2024)
- [Secure On-Device Video OOD Detection Without Backpropagation](https://mlsanity.com/paper/2503.06166/) (2025)
- [Video Quality Assessment: A Comprehensive Survey](https://mlsanity.com/paper/2412.04508/) (2024)
- [LLM4AD: Large Language Models for Autonomous Driving -- Concept, Review, Benchmark, Experiments, and Future Trends](https://mlsanity.com/paper/2410.15281/) (2024)
- [CWNet: Causal Wavelet Network for Low-Light Image Enhancement](https://mlsanity.com/paper/2507.10689/) (2025)
- [Vision Technologies with Applications in Traffic Surveillance Systems: A Holistic Survey](https://mlsanity.com/paper/2412.00348/) (2024)
- [GuideSR: Rethinking Guidance for One-Step High-Fidelity Diffusion-Based Super-Resolution](https://mlsanity.com/paper/2505.00687/) (2025)
- [From Enhancement to Understanding: Build a Generalized Bridge for Low-light Vision via Semantically Consistent Unsupervised Fine-tuning](https://mlsanity.com/paper/2507.08380/) (2025)
- [SPJFNet: Self-Mining Prior-Guided Joint Frequency Enhancement for Ultra-Efficient Dark Image Restoration](https://mlsanity.com/paper/2508.04041/) (2025)
- [RAW-Adapter: Adapting Pre-trained Visual Model to Camera RAW Images and A Benchmark](https://mlsanity.com/paper/2503.17027/) (2025)
- [TS-Diff: Two-Stage Diffusion Model for Low-Light RAW Image Enhancement](https://mlsanity.com/paper/2505.04281/) (2025)
- [A Novel Multi-layer Task-centric and Data Quality Framework for Autonomous Driving](https://mlsanity.com/paper/2506.17346/) (2025)
- [Adapting Large VLMs with Iterative and Manual Instructions for Generative Low-light Enhancement](https://mlsanity.com/paper/2507.18064/) (2025)
- [Optimizing 4D Lookup Table for Low-light Video Enhancement via Wavelet Priori](https://mlsanity.com/paper/2409.08585/) (2024)
- [LUMINA-Net: Low-light Upgrade through Multi-stage Illumination and Noise Adaptation Network for Image Enhancement](https://mlsanity.com/paper/2502.15186/) (2025)
- [Bidirectional Image-Event Guided Fusion Framework for Low-Light Image Enhancement](https://mlsanity.com/paper/2506.06120/) (2025)
- [BSMamba: Brightness and Semantic Modeling for Long-Range Interaction in Low-Light Image Enhancement](https://mlsanity.com/paper/2506.18346/) (2025)
- [CIVQLLIE: Causal Intervention with Vector Quantization for Low-Light Image Enhancement](https://mlsanity.com/paper/2508.03338/) (2025)
- [DRNet: All-in-One Image Restoration via Prior-Guided Dynamic Reparameterization](https://mlsanity.com/paper/2605.08627/) (2026)
- [FrozenDrive: Zero-Shot Text-Guided Driving Scene Generation and Data Augmentation with Parameter-Free Frozen Diffusion Model](https://mlsanity.com/paper/2606.20110/) (2026)
- [AngleRoCL: Angle-Robust Concept Learning for Physically View-Invariant T2I Adversarial Patches](https://mlsanity.com/paper/2506.09538/) (2025)
- [Automatic Labelling for Low-Light Pedestrian Detection](https://mlsanity.com/paper/2507.02513/) (2025)
- [A biologically inspired separable learning vision model for real-time traffic object perception in Dark](https://mlsanity.com/paper/2509.05012/) (2025)
- [Adapting SAM with Dynamic Similarity Graphs for Few-Shot Parameter-Efficient Small Dense Object Detection: A Case Study of Chickpea Pods in Field Conditions](https://mlsanity.com/paper/2509.25805/) (2025)
- [HistRetinex: Optimizing Retinex model in Histogram Domain for Efficient Low-Light Image Enhancement](https://mlsanity.com/paper/2510.21100/) (2025)
- [Unified Low-Light Traffic Image Enhancement via Multi-Stage Illumination Recovery and Adaptive Noise Suppression](https://mlsanity.com/paper/2511.17612/) (2025)
- [TPCNet: Triple physical constraints for Low-light Image Enhancement](https://mlsanity.com/paper/2511.22052/) (2025)

## Abstract (arXiv)

Vision-centric perception systems for autonomous driving have gained considerable attention recently due to their cost-effectiveness and scalability, especially compared to LiDAR-based systems. However, these systems often struggle in low-light conditions, potentially compromising their performance and safety. To address this, our paper introduces LightDiff, a domain-tailored framework designed to enhance the low-light image quality for autonomous driving applications. Specifically, we employ a multi-condition controlled diffusion model. LightDiff works without any human-collected paired data, leveraging a dynamic data degradation process instead. It incorporates a novel multi-condition adapter that adaptively controls the input weights from different modalities, including depth maps, RGB images, and text captions, to effectively illuminate dark scenes while maintaining context consistency. Furthermore, to align the enhanced images with the detection model's knowledge, LightDiff employs perception-specific scores as rewards to guide the diffusion training process through reinforcement learning. Extensive experiments on the nuScenes datasets demonstrate that LightDiff can significantly improve the performance of several state-of-the-art 3D detectors in night-time conditions while achieving high visual quality scores, highlighting its potential to safeguard autonomous driving.