mlsanityData to 14 Oct 2026

StyLitGAN: Prompting StyleGAN to Produce New Illumination Conditions

arXiv 2205.10351 ↗First posted 20 May 2022cs.CVcited by 4

Not on the map. Why?

Abstract (arXiv)

We propose a novel method, StyLitGAN, for relighting and resurfacing generated images in the absence of labeled data. Our approach generates images with realistic lighting effects, including cast shadows, soft shadows, inter-reflections, and glossy effects, without the need for paired or CGI data. StyLitGAN uses an intrinsic image method to decompose an image, followed by a search of the latent space of a pre-trained StyleGAN to identify a set of directions. By prompting the model to fix one component (e.g., albedo) and vary another (e.g., shading), we generate relighted images by adding the identified directions to the latent style codes. Quantitative metrics of change in albedo and lighting diversity allow us to choose effective directions using a forward selection process. Qualitative evaluation confirms the effectiveness of our method.

Key facts. Cited by 4 papers in the map (data through 14 Oct 2026). Builds on 35 of its 53 references that are in the map. First posted on arXiv 20 May 2022.

Builds on

35 of 53 references.

References · 35

  1. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  2. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,177
  3. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 6,013
  4. 2022Generative Adversarial Networkscited by 5,627
  5. 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,588
  6. 2016Perceptual Losses for Real-Time Style Transfer and Super-Resolutioncited by 2,694
  7. 2019Semantic Image Synthesis with Spatially-Adaptive Normalizationcited by 1,299
  8. 2021Alias-Free Generative Adversarial Networkscited by 1,184
  9. 2017Deep Image Priorcited by 1,098
  10. 2020Encoding in Style: a StyleGAN Encoder for Image-to-Image Translationcited by 727
  11. 2020In-Domain GAN Inversion for Real Image Editingcited by 420
  12. 2020InterFaceGAN: Interpreting the Disentangled Face Representation Learned by GANscited by 394
23 more
  1. 2021GAN Inversion: A Surveycited by 367
  2. 2020StyleFlow: Attribute-conditioned Exploration of StyleGAN-Generated Images using Conditional Continuous Normalizing Flowscited by 349
  3. 2020StyleSpace Analysis: Disentangled Controls for StyleGAN Image Generationcited by 332
  4. 2020Unsupervised Discovery of Interpretable Directions in the GAN Latent Spacecited by 289
  5. 2020StyleRig: Rigging StyleGAN for 3D Control over Portrait Imagescited by 268
  6. 2021EditGAN: High-Precision Semantic Image Editingcited by 156
  7. 2019Single Image Portrait Relightingcited by 152
  8. 2019Semantic Hierarchy Emerges in Deep Generative Representations for Scene Synthesiscited by 106
  9. 2021GAN-Control: Explicitly Controllable GANscited by 91
  10. 2019Learning Physics-guided Face Relighting under Directional Lightcited by 90
  11. 2018CGIntrinsics: Better Intrinsic Image Decomposition through Physically-Based Renderingcited by 85
  12. 2021A Shading-Guided Generative Implicit Model for Shape-Accurate 3D-Aware Image Synthesiscited by 72
  13. 2022Physically-Based Editing of Indoor Scene Lighting from a Single Imagecited by 62
  14. 2021JoJoGAN: One Shot Face Stylizationcited by 49
  15. 2016Learning Diverse Image Colorizationcited by 43
  16. 2017Self-Supervised Intrinsic Image Decompositioncited by 43
  17. 2022VoLux-GAN: A Generative Model for 3D Face Synthesis with HDRI Relightingcited by 37
  18. 2022BlobGAN: Spatially Disentangled Scene Representationscited by 35
  19. 2021Dual Contrastive Loss and Attention for GANscited by 30
  20. 2021A Light Stage on Every Deskcited by 21
  21. 2020Intrinsic Image Decomposition using Paradigmscited by 17
  22. 2021StyleGAN of All Trades: Image Manipulation with Only Pretrained StyleGANcited by 14
  23. 2020Cut-and-Paste Object Insertion by Enabling Deep Image Prior for Reshadingcited by 12

Built on by

Cited by year

Citing papers per year: 2023: 2; 2024: 2.

4 papers.

  1. 2023OBJECT 3DIT: Language-guided 3D-aware Image Editingcited by 63
  2. 2024A Diffusion Approach to Radiance Field Relighting using Multi-Illumination Synthesiscited by 37
  3. 2023UrbanIR: Large-Scale Urban Scene Inverse Rendering from a Single Videocited by 22
  4. 2024Retinex-Diffusion: On Controlling Illumination Conditions in Diffusion Models via Retinex Theorycited by 8