# ReLumix: Extending Image Relighting to Video via Video Diffusion Models

arXiv: https://arxiv.org/abs/2509.23769 · HTML: https://mlsanity.com/paper/2509.23769/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Placed in 3D, point cloud and scene generation, one of 14 directions in Diffusion models.
- Builds on 38 of its 44 references that are in the map.
- First posted on arXiv 28 Sep 2025.

## Placed in

- Diffusion models / [3D, point cloud and scene generation](https://mlsanity.com/diffusion/3d-point-cloud-and-scene-generation/)

## Builds on (most cited first, 30 of 38)

- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation](https://mlsanity.com/paper/2208.12242/) (2022)
- [Elucidating the Design Space of Diffusion-Based Generative Models](https://mlsanity.com/paper/2206.00364/) (2022)
- [InstructPix2Pix: Learning to Follow Image Editing Instructions](https://mlsanity.com/paper/2211.09800/) (2022)
- [Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasets](https://mlsanity.com/paper/2311.15127/) (2023)
- [Prompt-to-Prompt Image Editing with Cross Attention Control](https://mlsanity.com/paper/2208.01626/) (2022)
- [AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuning](https://mlsanity.com/paper/2307.04725/) (2023)
- [Imagic: Text-Based Real Image Editing with Diffusion Models](https://mlsanity.com/paper/2210.09276/) (2022)
- [ModelScope Text-to-Video Technical Report](https://mlsanity.com/paper/2308.06571/) (2023)
- [TokenFlow: Consistent Diffusion Features for Consistent Video Editing](https://mlsanity.com/paper/2307.10373/) (2023)
- [DragDiffusion: Harnessing Diffusion Models for Interactive Point-based Image Editing](https://mlsanity.com/paper/2306.14435/) (2023)
- [Drag Your GAN: Interactive Point-based Manipulation on the Generative Image Manifold](https://mlsanity.com/paper/2305.10973/) (2023)
- [IRON: Inverse Rendering by Optimizing Neural SDFs and Materials from Photometric Images](https://mlsanity.com/paper/2204.02232/) (2022)
- [NeILF: Neural Incident Light Field for Physically-based Material Estimation](https://mlsanity.com/paper/2203.07182/) (2022)
- [DiLightNet: Fine-grained Lighting Control for Diffusion-based Image Generation](https://mlsanity.com/paper/2402.11929/) (2024)
- [Neural Gaffer: Relighting Any Object via Diffusion](https://mlsanity.com/paper/2406.07520/) (2024)
- [Unified Shape and SVBRDF Recovery using Differentiable Monte Carlo Rendering](https://mlsanity.com/paper/2103.15208/) (2021)
- [SwitchLight: Co-design of Physics-driven Architecture and Pre-training Framework for Human Portrait Relighting](https://mlsanity.com/paper/2402.18848/) (2024)
- [Inverse Path Tracing for Joint Material and Lighting Estimation](https://mlsanity.com/paper/1903.07145/) (2019)
- [Relightful Harmonization: Lighting-aware Portrait Background Replacement](https://mlsanity.com/paper/2312.06886/) (2023)
- [DiffusionRenderer: Neural Inverse and Forward Rendering with Video Diffusion Models](https://mlsanity.com/paper/2501.18590/) (2025)
- [FlashTex: Fast Relightable Mesh Texturing with LightControlNet](https://mlsanity.com/paper/2402.13251/) (2024)
- [I2VEdit: First-Frame-Guided Video Editing via Image-to-Video Diffusion Models](https://mlsanity.com/paper/2405.16537/) (2024)
- [Neural-PBIR Reconstruction of Shape, Material, and Illumination](https://mlsanity.com/paper/2304.13445/) (2023)
- [UniRelight: Learning Joint Decomposition and Synthesis for Video Relighting](https://mlsanity.com/paper/2506.15673/) (2025)
- [Light-A-Video: Training-free Video Relighting via Progressive Light Fusion](https://mlsanity.com/paper/2502.08590/) (2025)
- [DifFRelight: Diffusion-Based Facial Performance Relighting](https://mlsanity.com/paper/2410.08188/) (2024)
- [RelightVid: Temporal-Consistent Diffusion Model for Video Relighting](https://mlsanity.com/paper/2501.16330/) (2025)
- [Lux Post Facto: Learning Portrait Performance Relighting with Conditional Video Diffusion and a Hybrid Dataset](https://mlsanity.com/paper/2503.14485/) (2025)
- [LoRA-Edit: Controllable First-Frame-Guided Video Editing via Mask-Aware LoRA Fine-Tuning](https://mlsanity.com/paper/2506.10082/) (2025)

## Built on by (most cited first, 1 of 1)

- [Generative Relightable Avatars](https://mlsanity.com/paper/2606.22718/) (2026)

## Abstract (arXiv)

Controlling illumination during video post-production is a crucial yet elusive goal in computational photography. Existing methods often lack flexibility, restricting users to certain relighting models. This paper introduces ReLumix, a novel framework that decouples the relighting algorithm from temporal synthesis, thereby enabling any image relighting technique to be seamlessly applied to video. Our approach reformulates video relighting into a simple yet effective two-stage process: (1) an artist relights a single reference frame using any preferred image-based technique (e.g., Diffusion Models, physics-based renderers); and (2) a fine-tuned stable video diffusion (SVD) model seamlessly propagates this target illumination throughout the sequence. To ensure temporal coherence and prevent artifacts, we introduce a gated cross-attention mechanism for smooth feature blending and a temporal bootstrapping strategy that harnesses SVD's powerful motion priors. Although trained on synthetic data, ReLumix shows competitive generalization to real-world videos. The method demonstrates significant improvements in visual fidelity, offering a scalable and versatile solution for dynamic lighting control.