mlsanityData to 7 Oct 2026

LightLab: Controlling Light Sources in Images with Diffusion Models

arXiv 2505.09608 ↗First posted 14 May 2025cs.CV cs.GRcited by 36

On the map: Diffusion models → 3D, point cloud and scene generation

Abstract (arXiv)

We present a simple, yet effective diffusion-based method for fine-grained, parametric control over light sources in an image. Existing relighting methods either rely on multiple input views to perform inverse rendering at inference time, or fail to provide explicit control over light changes. Our method fine-tunes a diffusion model on a small set of real raw photograph pairs, supplemented by synthetically rendered images at scale, to elicit its photorealistic prior for relighting. We leverage the linearity of light to synthesize image pairs depicting controlled light changes of either a target light source or ambient illumination. Using this data and an appropriate fine-tuning scheme, we train a model for precise illumination changes with explicit control over light intensity and color. Lastly, we show how our method can achieve compelling light editing results, and outperforms existing methods based on user preference.

Key facts. Cited by 36 papers in the map (data through 7 Oct 2026). Placed in 3D, point cloud and scene generation, one of 14 directions in Diffusion models. Builds on 20 of its 42 references that are in the map. First posted on arXiv 14 May 2025.

Builds on

20 of 42 references.

Closest ancestors · 6

Same direction, newest first

  1. 2024LumiNet: Latent Intrinsics Meets Diffusion Models for Indoor Scene Relighting29 Nov
  2. 2024Neural Gaffer: Relighting Any Object via Diffusion11 Jun
  3. 2024RGBleftrightarrowX: Image decomposition and synthesis using material- and lighting-aware diffusion models1 May
  4. 2024LightIt: Illumination Modeling and Control for Diffusion Models15 Mar
  5. 2024DiLightNet: Fine-grained Lighting Control for Diffusion-based Image Generation19 Feb
  6. 2023Relightful Harmonization: Lighting-aware Portrait Background Replacement11 Dec

Other references · 14

  1. 2016Image-to-Image Translation with Conditional Adversarial Networkscited by 4,510
  2. 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,290
  3. 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,595
  4. 2020Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domainscited by 2,170
  5. 2021Palette: Image-to-Image Diffusion Modelscited by 1,296
  6. 2023Objaverse-XL: A Universe of 10M+ 3D Objectscited by 767
  7. 2022PaLI: A Jointly-Scaled Multilingual Language-Image Modelcited by 762
  8. 2022Imagen Editor and EditBench: Advancing and Evaluating Text-Guided Image Inpaintingcited by 229
  9. 2019Inverse Rendering for Complex Indoor Scenes: Shape, Spatially-Varying Lighting and SVBRDF from a Single Imagecited by 202
  10. 2019Learning Physics-guided Face Relighting under Directional Lightcited by 90
  11. 2022StyleLight: HDR Panorama Generation for Lighting Estimation and Editingcited by 59
  12. 2019A Dataset of Multi-Illumination Images in the Wildcited by 45
2 more
  1. 2022OutCast: Outdoor Single-image Relighting with Cast Shadowscited by 30
  2. 2024ScribbleLight: Single Image Indoor Relighting with Scribblescited by 22

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 9; 2026: 27.

36 papers.

Sort:

By area:

  1. 2025Kontinuous Kontext: Continuous Strength Control for Instruction-based Image Editingcited by 15
  2. 2025Learning an Image Editing Model without Image Editing Pairscited by 13
  3. 2025RelightMaster: Precise Video Relighting with Multi-plane Light Imagescited by 8
  4. 2025PractiLight: Practical Light Control Using Foundational Diffusion Modelscited by 7
  5. 2026LuxRemix: Lighting Decomposition and Remixing for Indoor Scenescited by 7
  6. 2025Training-Free Text-Guided Color Editing with Multi-Modal Diffusion Transformercited by 5
  7. 2026GR3EN: Generative Relighting for 3D Environmentscited by 5
  8. 2026Continuous Control of Editing Models via Adaptive-Origin Guidancecited by 5
  9. 2025Coupled Diffusion Sampling for Training-Free Multi-View Image Editingcited by 4
  10. 2025NeAR: Coupled Neural Asset-Renderer Stackcited by 4
  11. 2025UniLight: A Unified Representation for Lightingcited by 3
  12. 2026GeoRelight: Learning Joint Geometrical Relighting and Reconstruction with Flexible Multi-Modal Diffusion Transformerscited by 3
24 more
  1. 2025GimbalDiffusion: Gravity-Aware Camera Control for Video Generationcited by 2
  2. 2026Light4D: Training-Free Extreme Viewpoint 4D Video Relightingcited by 2
  3. 2026EAG-PT: Emission-Aware Gaussians and Path Tracing for Diffuse Indoor Scene Reconstruction and Editingcited by 1
  4. 2026Learning Latent Proxies for Controllable Single-Image Relightingcited by 1
  5. 2026TokenLight: Precise Lighting Control in Images using Attribute Tokenscited by 1
  6. 2026Relit-LiVE: Relight Video by Jointly Learning Environment Videocited by 1
  7. 2026Decoupled Illumination Priors for Spatially Controllable Multi-View Indoor Scene Relightingcited by 1
  8. 2026Alterbute: Editing Intrinsic Attributes of Objects in Imagescited by 0
  9. 2026SyncLight: Single-Edit Multi-View Relightingcited by 0
  10. 2026Relighting as a Probe of Visual Priors via Augmented Latent Intrinsicscited by 0
  11. 2026CamLit: Unified Video Diffusion with Explicit Camera and Lighting Controlcited by 0
  12. 2026LightMover: Generative Light Movement with Color and Intensity Controlscited by 0
  13. 2026PIXLRelight: Controllable Relighting via Intrinsic Conditioningcited by 0
  14. 2026BodyReLux: Temporally Consistent Full-Body Video Relightingcited by 0
  15. 2026Pixel Cube: Diffusion-based Portrait Video Relighting Through Realistic Lighting Reproductioncited by 0
  16. 2026TRON: Tracing Rays to Orchestrate a Neural Renderer for 3D Gaussian Reconstructionscited by 0
  17. 2026Thinking in Boxes: 3D Editing in Real Images Made Easycited by 0
  18. 2026Lighting-Consistent Object Transfer Across Radiance Fieldscited by 0
  19. 2026Do Image Editing Models Understand Lighting?cited by 0
  20. 2026FreeLit: Paired-Free Indoor Relighting via Physics-Guided Diffusioncited by 0
  21. 2026Consistent Feature Transport for Image Relightingcited by 0
  22. 2026Representation Learning in Diffusion and Flow-based Model: An Application Aspectcited by 0
  23. 2026LightBridge: Feed-Forward Generative Relighting for 3D Gaussian Splattingcited by 0
  24. 2026Geometry Without Coordinates: LiDAR Diffusion as a 3D Feature Bridgecited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026Learning Illumination Control in Diffusion Models
  2. 2025TransLight: Image-Guided Customized Lighting Control with Generative Decoupling
  3. 2026LightCtrl: Training-free Controllable Video Relighting
  4. 2026PI-Light: Physics-Inspired Diffusion for Full-Image Relighting
  5. 2025SynthLight: Portrait Relighting with Diffusion Model by Learning to Re-render Synthetic Faces
  6. 2024A Diffusion Approach to Radiance Field Relighting using Multi-Illumination Synthesis
  7. 2026LoRA Direction Extraction for Controllable Light Toggling in FLUX.1 Kontext
  8. 2025ReLumix: Extending Image Relighting to Video via Video Diffusion Models