mlsanityData to 14 Oct 2026

SwitchLight: Co-design of Physics-driven Architecture and Pre-training Framework for Human Portrait Relighting

arXiv 2402.18848 ↗First posted 29 Feb 2024cs.CVcited by 67

Not on the map. Why?

Cited by 21 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

We introduce a co-designed approach for human portrait relighting that combines a physics-guided architecture with a pre-training framework. Drawing on the Cook-Torrance reflectance model, we have meticulously configured the architecture design to precisely simulate light-surface interactions. Furthermore, to overcome the limitation of scarce high-quality lightstage data, we have developed a self-supervised pre-training strategy. This novel combination of accurate physical modeling and expanded training dataset establishes a new benchmark in relighting realism.

Key facts. Cited by 67 papers in the map (data through 14 Oct 2026). Builds on 36 of its 56 references that are in the map. First posted on arXiv 29 Feb 2024.

Builds on

36 of 56 references.

References · 36

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  3. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 12,049
  4. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,255
  5. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,513
  6. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
  7. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 6,013
  8. 2016Image-to-Image Translation with Conditional Adversarial Networkscited by 4,527
  9. 2016Perceptual Losses for Real-Time Style Transfer and Super-Resolutioncited by 2,694
  10. 2021BEiT: BERT Pre-Training of Image Transformerscited by 2,375
  11. 2018Unsupervised Representation Learning by Predicting Image Rotationscited by 1,591
  12. 2016Context Encoders: Feature Learning by Inpaintingcited by 1,428
24 more
  1. 2016Unsupervised Learning of Visual Representations by Solving Jigsaw Puzzlescited by 1,382
  2. 2016Colorful Image Colorizationcited by 1,114
  3. 2015Unsupervised Visual Representation Learning by Context Predictioncited by 1,066
  4. 2020Pre-Trained Image Processing Transformercited by 893
  5. 2023ConvNeXt V2: Co-designing and Scaling ConvNets with Masked Autoencoderscited by 712
  6. 2018Image Inpainting for Irregular Holes Using Partial Convolutionscited by 505
  7. 2022Activating More Pixels in Image Super-Resolution Transformercited by 438
  8. 2020Real-Time High-Resolution Background Mattingcited by 173
  9. 2019Single Image Portrait Relightingcited by 152
  10. 2017SfSNet: Learning Shape, Reflectance and Illuminance of Faces in the Wildcited by 110
  11. 2020Background Matting: The World is Your Green Screencited by 105
  12. 2023What Do Self-Supervised Vision Transformers Learn?cited by 93
  13. 2019Learning Physics-guided Face Relighting under Directional Lightcited by 90
  14. 2022Learning to Relight Portrait Images via a Virtual Light Stage and Synthetic-to-Real Adaptationcited by 67
  15. 2021On Efficient Transformer-Based Image Pre-training for Low-Level Visioncited by 64
  16. 2019Boundless: Generative Adversarial Networks for Image Extensioncited by 62
  17. 2022Face Relighting with Geometrically Consistent Shadowscited by 43
  18. 2021Towards High Fidelity Face Relighting with Realistic Shadowscited by 42
  19. 2022Revisiting Image Pyramid Structure for High Resolution Salient Object Detectioncited by 36
  20. 2022Geometry-aware Single-image Full-body Human Relightingcited by 34
  21. 2022SunStage: Portrait Reconstruction and Relighting using the Sun as a Light Stagecited by 30
  22. 2021A Light Stage on Every Deskcited by 21
  23. 2023DiFaReli++: Diffusion Face Relighting with Consistent Cast Shadowscited by 14
  24. 2023Controllable Light Diffusion for Portraitscited by 11

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 10; 2025: 32; 2026: 25.

67 papers.

Sort:

By area:

  1. 2024URAvatar: Universal Relightable Gaussian Codec Avatarscited by 43
  2. 2024LumiNet: Latent Intrinsics Meets Diffusion Models for Indoor Scene Relightingcited by 38
  3. 2024GAF: Gaussian Avatar Reconstruction from Monocular Videos via Multi-view Diffusioncited by 36
  4. 2025LBM: Latent Bridge Matching for Fast Image-to-Image Translationcited by 36
  5. 2024DifFRelight: Diffusion-Based Facial Performance Relightingcited by 35
  6. 2025Lux Post Facto: Learning Portrait Performance Relighting with Conditional Video Diffusion and a Hybrid Datasetcited by 28
  7. 2025VidCRAFT3: Camera, Object, and Lighting Control for Image-to-Video Generationcited by 25
  8. 2025HRAvatar: High-Quality and Relightable Gaussian Head Avatarcited by 25
  9. 2024ScribbleLight: Single Image Indoor Relighting with Scribblescited by 24
  10. 2025SynthLight: Portrait Relighting with Diffusion Model by Learning to Re-render Synthetic Facescited by 24
  11. 2025Relightable Full-Body Gaussian Codec Avatarscited by 22
  12. 2025Ouroboros: Single-step Diffusion Models for Cycle-consistent Forward and Inverse Renderingcited by 16
38 more
  1. 2025Lumen: Consistent Video Relighting and Harmonious Background Replacement with Video Generative Modelscited by 13
  2. 2024Surfel-based Gaussian Inverse Rendering for Fast and Relightable Dynamic Human Reconstruction from Monocular Videocited by 12
  3. 2025Comprehensive Relighting: Generalizable and Consistent Monocular Human Relighting and Harmonizationcited by 12
  4. 2025UniLumos: Fast and Unified Image and Video Relighting with Physics-Plausible Feedbackcited by 12
  5. 2025Light-X: Generative 4D Video Rendering with Camera and Illumination Controlcited by 12
  6. 2025TC-Light: Temporally Coherent Generative Rendering for Realistic World Transfercited by 11
  7. 2024Generative Portrait Shadow Removalcited by 10
  8. 2026LuxRemix: Lighting Decomposition and Remixing for Indoor Scenescited by 9
  9. 2025DreamLight: Towards Harmonious and Consistent Image Relightingcited by 8
  10. 2025IllumiCraft: Unified Geometry and Illumination Diffusion for Controllable Video Generationcited by 7
  11. 2025HumanOLAT: A Large-Scale Dataset for Full-Body Human Relighting and Novel-View Synthesiscited by 7
  12. 2026LiveLight: Real-time Streaming Video Relighting with Interactive Controlcited by 7
  13. 2024Interactive Rendering of Relightable and Animatable Gaussian Avatarscited by 6
  14. 2025FlowPortal: Residual-Corrected Flow for Training-Free Video Relighting and Background Replacementcited by 6
  15. 2025Parametric Shadow Control for Portrait Generation in Text-to-Image Diffusion Modelscited by 5
  16. 2025ANYPORTAL: Zero-Shot Consistent Video Background Replacementcited by 5
  17. 2025Generalizable and Relightable Gaussian Splatting for Human Novel View Synthesiscited by 4
  18. 2025VGGTFace: Topologically Consistent Facial Geometry Reconstruction in the Wildcited by 4
  19. 2026Sapiens2cited by 4
  20. 2024LumiSculpt: Enabling Consistent Portrait Lighting in Video Generationcited by 3
  21. 2026LightCtrl: Training-free Controllable Video Relightingcited by 3
  22. 2026GeoRelight: Learning Joint Geometrical Relighting and Reconstruction with Flexible Multi-Modal Diffusion Transformerscited by 3
  23. 2025Beyond Reconstruction: A Physics Based Neural Deferred Shader for Photo-realistic Renderingcited by 2
  24. 2025Total-Editing: Head Avatar with Editable Appearance, Motion, and Lightingcited by 2
  25. 2025Training-Free Multi-View Extension of IC-Light for Textual Position-Aware Scene Relightingcited by 2
  26. 2025Refracting Reality: Generating Images with Realistic Transparent Objectscited by 2
  27. 2025Relightable Holoported Characters: Capturing and Relighting Dynamic Human Performance from Sparse Viewscited by 2
  28. 2026Light4D: Training-Free Extreme Viewpoint 4D Video Relightingcited by 2
  29. 2024MixLight: Borrowing the Best of both Spherical Harmonics and Gaussian Modelscited by 1
  30. 2025Joint Learning of Depth and Appearance for Portrait Image Animationcited by 1
  31. 2025HumanMaterial: Human Material Estimation from a Single Image via Progressive Trainingcited by 1
  32. 2025ReLumix: Extending Image Relighting to Video via Video Diffusion Modelscited by 1
  33. 2025WildCap: Facial Albedo Capture in the Wild via Hybrid Inverse Renderingcited by 1
  34. 2025POLAR: A Portrait OLAT Dataset and Generative Framework for Illumination-Aware Face Modelingcited by 1
  35. 2026TokenLight: Precise Lighting Control in Images using Attribute Tokenscited by 1
  36. 2026Decoupled Illumination Priors for Spatially Controllable Multi-View Indoor Scene Relightingcited by 1
  37. 2025Using Gaussian Splats to Create High-Fidelity Facial Geometry and Texturecited by 0
  38. 2025A Convolutional Neural Deferred Shader for Physics Based Renderingcited by 0

50 most cited of 67 All 67 (JSON)