# RelightVid: Temporal-Consistent Diffusion Model for Video Relighting

arXiv: https://arxiv.org/abs/2501.16330 · HTML: https://mlsanity.com/paper/2501.16330/

## Key facts

- Cited by 29 papers in the map (data through 14 Oct 2026).
- Placed in 3D, point cloud and scene generation, one of 14 directions in Diffusion models.
- Builds on 16 of its 57 references that are in the map.
- First posted on arXiv 27 Jan 2025.

## Placed in

- Diffusion models / [3D, point cloud and scene generation](https://mlsanity.com/diffusion/3d-point-cloud-and-scene-generation/)

## Builds on (most cited first, 16 of 16)

- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [DreamFusion: Text-to-3D using 2D Diffusion](https://mlsanity.com/paper/2209.14988/) (2022)
- [Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasets](https://mlsanity.com/paper/2311.15127/) (2023)
- [Prompt-to-Prompt Image Editing with Cross Attention Control](https://mlsanity.com/paper/2208.01626/) (2022)
- [CogVideoX: Text-to-Video Diffusion Models with An Expert Transformer](https://mlsanity.com/paper/2408.06072/) (2024)
- [Decoupled Weight Decay Regularization](https://mlsanity.com/paper/1711.05101/) (2017)
- [AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuning](https://mlsanity.com/paper/2307.04725/) (2023)
- [DiffEdit: Diffusion-based semantic image editing with mask guidance](https://mlsanity.com/paper/2210.11427/) (2022)
- [TokenFlow: Consistent Diffusion Features for Consistent Video Editing](https://mlsanity.com/paper/2307.10373/) (2023)
- [MotionClone: Training-Free Motion Cloning for Controllable Video Generation](https://mlsanity.com/paper/2406.05338/) (2024)
- [Neural Gaffer: Relighting Any Object via Diffusion](https://mlsanity.com/paper/2406.07520/) (2024)
- [Consistent Video-to-Video Transfer Using Synthetic Dataset](https://mlsanity.com/paper/2311.00213/) (2023)
- [ReVideo: Remake a Video with Motion and Content Control](https://mlsanity.com/paper/2405.13865/) (2024)
- [X-Prompt: Towards Universal In-Context Image Generation in Auto-Regressive Vision Language Foundation Models](https://mlsanity.com/paper/2412.01824/) (2024)
- [ByTheWay: Boost Your Text-to-Video Generation Model to Higher Quality in a Training-free Way](https://mlsanity.com/paper/2410.06241/) (2024)
- [Bootstrap3D: Improving Multi-view Diffusion Model with Synthetic Data](https://mlsanity.com/paper/2406.00093/) (2024)

## Built on by (most cited first, 29 of 29)

- [VidCRAFT3: Camera, Object, and Lighting Control for Image-to-Video Generation](https://mlsanity.com/paper/2502.07531/) (2025)
- [LightSwitch: Multi-view Relighting with Material-guided Diffusion](https://mlsanity.com/paper/2508.06494/) (2025)
- [Lumen: Consistent Video Relighting and Harmonious Background Replacement with Video Generative Models](https://mlsanity.com/paper/2508.12945/) (2025)
- [UniLumos: Fast and Unified Image and Video Relighting with Physics-Plausible Feedback](https://mlsanity.com/paper/2511.01678/) (2025)
- [Light-X: Generative 4D Video Rendering with Camera and Illumination Control](https://mlsanity.com/paper/2512.05115/) (2025)
- [TC-Light: Temporally Coherent Generative Rendering for Realistic World Transfer](https://mlsanity.com/paper/2506.18904/) (2025)
- [RelightMaster: Precise Video Relighting with Multi-plane Light Images](https://mlsanity.com/paper/2511.06271/) (2025)
- [IllumiCraft: Unified Geometry and Illumination Diffusion for Controllable Video Generation](https://mlsanity.com/paper/2506.03150/) (2025)
- [LiveLight: Real-time Streaming Video Relighting with Interactive Control](https://mlsanity.com/paper/2608.01771/) (2026)
- [FlowPortal: Residual-Corrected Flow for Training-Free Video Relighting and Background Replacement](https://mlsanity.com/paper/2511.18346/) (2025)
- [ANYPORTAL: Zero-Shot Consistent Video Background Replacement](https://mlsanity.com/paper/2509.07472/) (2025)
- [GR3EN: Generative Relighting for 3D Environments](https://mlsanity.com/paper/2601.16272/) (2026)
- [Taming generative video models for zero-shot optical flow extraction](https://mlsanity.com/paper/2507.09082/) (2025)
- [LightCtrl: Training-free Controllable Video Relighting](https://mlsanity.com/paper/2603.27083/) (2026)
- [Training-Free Multi-View Extension of IC-Light for Textual Position-Aware Scene Relighting](https://mlsanity.com/paper/2511.13684/) (2025)
- [Light4D: Training-Free Extreme Viewpoint 4D Video Relighting](https://mlsanity.com/paper/2602.11769/) (2026)
- [ReLumix: Extending Image Relighting to Video via Video Diffusion Models](https://mlsanity.com/paper/2509.23769/) (2025)
- [TokenLight: Precise Lighting Control in Images using Attribute Tokens](https://mlsanity.com/paper/2604.15310/) (2026)
- [Relit-LiVE: Relight Video by Jointly Learning Environment Video](https://mlsanity.com/paper/2605.06658/) (2026)
- [ID-V2V: Identity-Preserving Video Restylization](https://mlsanity.com/paper/2607.22830/) (2026)
- [SyncLight: Single-Edit Multi-View Relighting](https://mlsanity.com/paper/2601.16981/) (2026)
- [GO-Renderer: Generative Object Rendering with 3D-aware Controllable Video Diffusion Models](https://mlsanity.com/paper/2603.23246/) (2026)
- [VibeFlow: Versatile Video Chroma-Lux Editing through Self-Supervised Learning](https://mlsanity.com/paper/2604.13425/) (2026)
- [FusionRelight: Relighting Portraits in Real Time via Hybrid Domain Knowledge Fusion](https://mlsanity.com/paper/2604.23094/) (2026)
- [BodyReLux: Temporally Consistent Full-Body Video Relighting](https://mlsanity.com/paper/2605.21766/) (2026)
- [HarmoVid: Relightful Video Portrait Harmonization](https://mlsanity.com/paper/2605.28811/) (2026)
- [Pixel Cube: Diffusion-based Portrait Video Relighting Through Realistic Lighting Reproduction](https://mlsanity.com/paper/2606.02919/) (2026)
- [Cinematic Compositing Using Character-Environment-Harmonized Video Generation Models](https://mlsanity.com/paper/2606.20233/) (2026)
- [FreeCHROMA: Generative Chroma Keying from Unaligned Background References](https://mlsanity.com/paper/2610.14445/) (2026)

## Abstract (arXiv)

Diffusion models have demonstrated remarkable success in image generation and editing, with recent advancements enabling albedo-preserving image relighting. However, applying these models to video relighting remains challenging due to the lack of paired video relighting datasets and the high demands for output fidelity and temporal consistency, further complicated by the inherent randomness of diffusion models. To address these challenges, we introduce RelightVid, a flexible framework for video relighting that can accept background video, text prompts, or environment maps as relighting conditions. Trained on in-the-wild videos with carefully designed illumination augmentations and rendered videos under extreme dynamic lighting, RelightVid achieves arbitrary video relighting with high temporal consistency without intrinsic decomposition while preserving the illumination priors of its image backbone.