# LightLab: Controlling Light Sources in Images with Diffusion Models

arXiv: https://arxiv.org/abs/2505.09608 · HTML: https://mlsanity.com/paper/2505.09608/

## Key facts

- Cited by 38 papers in the map (data through 14 Oct 2026).
- Placed in 3D, point cloud and scene generation, one of 14 directions in Diffusion models.
- Builds on 20 of its 42 references that are in the map.
- First posted on arXiv 14 May 2025.

## Placed in

- Diffusion models / [3D, point cloud and scene generation](https://mlsanity.com/diffusion/3d-point-cloud-and-scene-generation/)

## Builds on (most cited first, 20 of 20)

- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis](https://mlsanity.com/paper/2307.01952/) (2023)
- [GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models](https://mlsanity.com/paper/2112.10741/) (2021)
- [Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains](https://mlsanity.com/paper/2006.10739/) (2020)
- [Palette: Image-to-Image Diffusion Models](https://mlsanity.com/paper/2111.05826/) (2021)
- [Objaverse-XL: A Universe of 10M+ 3D Objects](https://mlsanity.com/paper/2307.05663/) (2023)
- [PaLI: A Jointly-Scaled Multilingual Language-Image Model](https://mlsanity.com/paper/2209.06794/) (2022)
- [Imagen Editor and EditBench: Advancing and Evaluating Text-Guided Image Inpainting](https://mlsanity.com/paper/2212.06909/) (2022)
- [Inverse Rendering for Complex Indoor Scenes: Shape, Spatially-Varying Lighting and SVBRDF from a Single Image](https://mlsanity.com/paper/1905.02722/) (2019)
- [RGBleftrightarrowX: Image decomposition and synthesis using material- and lighting-aware diffusion models](https://mlsanity.com/paper/2405.00666/) (2024)
- [DiLightNet: Fine-grained Lighting Control for Diffusion-based Image Generation](https://mlsanity.com/paper/2402.11929/) (2024)
- [Neural Gaffer: Relighting Any Object via Diffusion](https://mlsanity.com/paper/2406.07520/) (2024)
- [Learning Physics-guided Face Relighting under Directional Light](https://mlsanity.com/paper/1906.03355/) (2019)
- [StyleLight: HDR Panorama Generation for Lighting Estimation and Editing](https://mlsanity.com/paper/2207.14811/) (2022)
- [Relightful Harmonization: Lighting-aware Portrait Background Replacement](https://mlsanity.com/paper/2312.06886/) (2023)
- [LightIt: Illumination Modeling and Control for Diffusion Models](https://mlsanity.com/paper/2403.10615/) (2024)
- [A Dataset of Multi-Illumination Images in the Wild](https://mlsanity.com/paper/1910.08131/) (2019)
- [LumiNet: Latent Intrinsics Meets Diffusion Models for Indoor Scene Relighting](https://mlsanity.com/paper/2412.00177/) (2024)
- [OutCast: Outdoor Single-image Relighting with Cast Shadows](https://mlsanity.com/paper/2204.09341/) (2022)
- [ScribbleLight: Single Image Indoor Relighting with Scribbles](https://mlsanity.com/paper/2411.17696/) (2024)

## Built on by (most cited first, 30 of 38)

- [Kontinuous Kontext: Continuous Strength Control for Instruction-based Image Editing](https://mlsanity.com/paper/2510.08532/) (2025)
- [Learning an Image Editing Model without Image Editing Pairs](https://mlsanity.com/paper/2510.14978/) (2025)
- [LuxRemix: Lighting Decomposition and Remixing for Indoor Scenes](https://mlsanity.com/paper/2601.15283/) (2026)
- [RelightMaster: Precise Video Relighting with Multi-plane Light Images](https://mlsanity.com/paper/2511.06271/) (2025)
- [PractiLight: Practical Light Control Using Foundational Diffusion Models](https://mlsanity.com/paper/2509.01837/) (2025)
- [Training-Free Text-Guided Color Editing with Multi-Modal Diffusion Transformer](https://mlsanity.com/paper/2508.09131/) (2025)
- [NeAR: Coupled Neural Asset-Renderer Stack](https://mlsanity.com/paper/2511.18600/) (2025)
- [GR3EN: Generative Relighting for 3D Environments](https://mlsanity.com/paper/2601.16272/) (2026)
- [Continuous Control of Editing Models via Adaptive-Origin Guidance](https://mlsanity.com/paper/2602.03826/) (2026)
- [Coupled Diffusion Sampling for Training-Free Multi-View Image Editing](https://mlsanity.com/paper/2510.14981/) (2025)
- [UniLight: A Unified Representation for Lighting](https://mlsanity.com/paper/2512.04267/) (2025)
- [GeoRelight: Learning Joint Geometrical Relighting and Reconstruction with Flexible Multi-Modal Diffusion Transformers](https://mlsanity.com/paper/2604.20715/) (2026)
- [GimbalDiffusion: Gravity-Aware Camera Control for Video Generation](https://mlsanity.com/paper/2512.09112/) (2025)
- [Light4D: Training-Free Extreme Viewpoint 4D Video Relighting](https://mlsanity.com/paper/2602.11769/) (2026)
- [EAG-PT: Emission-Aware Gaussians and Path Tracing for Diffuse Indoor Scene Reconstruction and Editing](https://mlsanity.com/paper/2601.23065/) (2026)
- [Learning Latent Proxies for Controllable Single-Image Relighting](https://mlsanity.com/paper/2603.15555/) (2026)
- [TokenLight: Precise Lighting Control in Images using Attribute Tokens](https://mlsanity.com/paper/2604.15310/) (2026)
- [Relit-LiVE: Relight Video by Jointly Learning Environment Video](https://mlsanity.com/paper/2605.06658/) (2026)
- [Do Image Editing Models Understand Lighting?](https://mlsanity.com/paper/2606.26738/) (2026)
- [Decoupled Illumination Priors for Spatially Controllable Multi-View Indoor Scene Relighting](https://mlsanity.com/paper/2607.08879/) (2026)
- [Geometry Without Coordinates: LiDAR Diffusion as a 3D Feature Bridge](https://mlsanity.com/paper/2609.10322/) (2026)
- [Alterbute: Editing Intrinsic Attributes of Objects in Images](https://mlsanity.com/paper/2601.10714/) (2026)
- [SyncLight: Single-Edit Multi-View Relighting](https://mlsanity.com/paper/2601.16981/) (2026)
- [Relighting as a Probe of Visual Priors via Augmented Latent Intrinsics](https://mlsanity.com/paper/2602.01391/) (2026)
- [CamLit: Unified Video Diffusion with Explicit Camera and Lighting Control](https://mlsanity.com/paper/2603.14241/) (2026)
- [LightMover: Generative Light Movement with Color and Intensity Controls](https://mlsanity.com/paper/2603.27209/) (2026)
- [PIXLRelight: Controllable Relighting via Intrinsic Conditioning](https://mlsanity.com/paper/2605.18735/) (2026)
- [BodyReLux: Temporally Consistent Full-Body Video Relighting](https://mlsanity.com/paper/2605.21766/) (2026)
- [Pixel Cube: Diffusion-based Portrait Video Relighting Through Realistic Lighting Reproduction](https://mlsanity.com/paper/2606.02919/) (2026)
- [TRON: Tracing Rays to Orchestrate a Neural Renderer for 3D Gaussian Reconstructions](https://mlsanity.com/paper/2606.11314/) (2026)

## Abstract (arXiv)

We present a simple, yet effective diffusion-based method for fine-grained, parametric control over light sources in an image. Existing relighting methods either rely on multiple input views to perform inverse rendering at inference time, or fail to provide explicit control over light changes. Our method fine-tunes a diffusion model on a small set of real raw photograph pairs, supplemented by synthetically rendered images at scale, to elicit its photorealistic prior for relighting. We leverage the linearity of light to synthesize image pairs depicting controlled light changes of either a target light source or ambient illumination. Using this data and an appropriate fine-tuning scheme, we train a model for precise illumination changes with explicit control over light intensity and color. Lastly, we show how our method can achieve compelling light editing results, and outperforms existing methods based on user preference.