# FusionRelight: Relighting Portraits in Real Time via Hybrid Domain Knowledge Fusion

arXiv: https://arxiv.org/abs/2604.23094 · HTML: https://mlsanity.com/paper/2604.23094/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 25 of its 34 references that are in the map.
- First posted on arXiv 25 Apr 2026.

## Builds on (most cited first, 25 of 25)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [A ConvNet for the 2020s](https://mlsanity.com/paper/2201.03545/) (2022)
- [Knowledge Distillation: A Survey](https://mlsanity.com/paper/2006.05525/) (2020)
- [Self-training with Noisy Student improves ImageNet classification](https://mlsanity.com/paper/1911.04252/) (2019)
- [Albumentations: fast and flexible image augmentations](https://mlsanity.com/paper/1809.06839/) (2018)
- [One-Shot Free-View Neural Talking-Head Synthesis for Video Conferencing](https://mlsanity.com/paper/2011.15126/) (2020)
- [Advancing High-Resolution Video-Language Representation with Large-Scale Video Transcriptions](https://mlsanity.com/paper/2111.10337/) (2021)
- [Stable and Controllable Neural Texture Synthesis and Style Transfer Using Histogram Losses](https://mlsanity.com/paper/1701.08893/) (2017)
- [Learning to Relight Portrait Images via a Virtual Light Stage and Synthetic-to-Real Adaptation](https://mlsanity.com/paper/2209.10510/) (2022)
- [SwitchLight: Co-design of Physics-driven Architecture and Pre-training Framework for Human Portrait Relighting](https://mlsanity.com/paper/2402.18848/) (2024)
- [HistoGAN: Controlling Colors of GAN-Generated and Real Images via Color Histograms](https://mlsanity.com/paper/2011.11731/) (2020)
- [How to Train Neural Networks for Flare Removal](https://mlsanity.com/paper/2011.12485/) (2020)
- [Neural Video Portrait Relighting in Real-time via Consistency Modeling](https://mlsanity.com/paper/2104.00484/) (2021)
- [UniRelight: Learning Joint Decomposition and Synthesis for Video Relighting](https://mlsanity.com/paper/2506.15673/) (2025)
- [RelightVid: Temporal-Consistent Diffusion Model for Video Relighting](https://mlsanity.com/paper/2501.16330/) (2025)
- [Lux Post Facto: Learning Portrait Performance Relighting with Conditional Video Diffusion and a Hybrid Dataset](https://mlsanity.com/paper/2503.14485/) (2025)
- [Lite2Relight: 3D-aware Single Image Portrait Relighting](https://mlsanity.com/paper/2407.10487/) (2024)
- [SynthLight: Portrait Relighting with Diffusion Model by Learning to Re-render Synthetic Faces](https://mlsanity.com/paper/2501.09756/) (2025)
- [Real-time 3D-aware Portrait Video Relighting](https://mlsanity.com/paper/2410.18355/) (2024)
- [EdgeRelight360: Text-Conditioned 360-Degree HDR Image Generation for Real-Time On-Device Video Portrait Relighting](https://mlsanity.com/paper/2404.09918/) (2024)
- [PI-Light: Physics-Inspired Diffusion for Full-Image Relighting](https://mlsanity.com/paper/2601.22135/) (2026)
- [Yesnt: Are Diffusion Relighting Models Ready for Capture Stage Compositing? A Hybrid Alternative to Bridge the Gap](https://mlsanity.com/paper/2510.23494/) (2025)

## Abstract (arXiv)

Portrait relighting is a low-level vision problem in which physically plausible illumination transfer, identity preservation, and compact real-time inference must be considered together. Iterative diffusion-style methods can synthesize fine detail, but stochastic inference and cost complicate deterministic live video creation; physically grounded relighting preserves identity, but controlled synthetic or light-stage supervision transfers poorly to unconstrained cameras. We present Hybrid Domain Knowledge Fusion (HDKF), a relighting-specific training framework that learns complementary physics, reflectance, and realism priors from synthetic, One-Light-at-A-Time (OLAT), and in-the-wild data, then distills their source-routed supervision into a compact student with clean teacher labels and degraded student inputs. The framework is trained with pixel-aligned RGB, albedo, and normal supervision, providing a simulation substrate for physically grounded low-level relighting. On a held-out OLAT benchmark, HDKF obtains the best MSE, PSNR, and SSIM among evaluated methods while remaining competitive in LPIPS. The distilled model runs in real time at 512x512, reaching 11.89 ms on an RTX 2060 and 1.82 ms on an RTX 4090.