mlsanityData to 14 Oct 2026

SunStage: Portrait Reconstruction and Relighting using the Sun as a Light Stage

arXiv 2204.03648 ↗First posted 7 Apr 2022cs.CVcited by 30

Not on the map. Why?

Abstract (arXiv)

A light stage uses a series of calibrated cameras and lights to capture a subject's facial appearance under varying illumination and viewpoint. This captured information is crucial for facial reconstruction and relighting. Unfortunately, light stages are often inaccessible: they are expensive and require significant technical expertise for construction and operation. In this paper, we present SunStage: a lightweight alternative to a light stage that captures comparable data using only a smartphone camera and the sun. Our method only requires the user to capture a selfie video outdoors, rotating in place, and uses the varying angles between the sun and the face as guidance in joint reconstruction of facial geometry, reflectance, camera pose, and lighting parameters. Despite the in-the-wild un-calibrated setting, our approach is able to reconstruct detailed facial appearance and geometry, enabling compelling effects such as relighting, novel view synthesis, and reflectance editing. Results and interactive demos are available at https://sunstage.cs.washington.edu/.

Key facts. Cited by 30 papers in the map (data through 14 Oct 2026). Builds on 32 of its 69 references that are in the map. First posted on arXiv 7 Apr 2022.

Builds on

32 of 69 references.

References · 32

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,177
  3. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 6,013
  4. 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,588
  5. 2016Perceptual Losses for Real-Time Style Transfer and Super-Resolutioncited by 2,694
  6. 2019Deep High-Resolution Representation Learning for Visual Recognitioncited by 1,420
  7. 2019Semantic Image Synthesis with Spatially-Adaptive Normalizationcited by 1,299
  8. 2017High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANscited by 1,179
  9. 2020Accelerating 3D Deep Learning with PyTorch3Dcited by 784
  10. 2018BiSeNet: Bilateral Segmentation Network for Real-time Semantic Segmentationcited by 645
  11. 2020Learning an Animatable Detailed 3D Face Model from In-The-Wild Imagescited by 477
  12. 2020BiSeNet V2: Bilateral Network with Guided Aggregation for Real-time Semantic Segmentationcited by 294
20 more
  1. 20193D Morphable Face Models -- Past, Present and Futurecited by 270
  2. 2018Generating 3D faces using Convolutional Mesh Autoencoderscited by 268
  3. 2018Nonlinear 3D Face Morphable Modelcited by 157
  4. 2019Single Image Portrait Relightingcited by 152
  5. 2021Robust High-Resolution Video Matting with Temporal Guidancecited by 137
  6. 2017SfSNet: Learning Shape, Reflectance and Illuminance of Faces in the Wildcited by 110
  7. 2019Learning Physics-guided Face Relighting under Directional Lightcited by 90
  8. 2020Neural Light Transport for Relighting and View Synthesiscited by 73
  9. 2019Combining 3D Morphable Models: A Large scale Face-and-Head Modelcited by 49
  10. 2021Towards High Fidelity Monocular Face Reconstruction with Rich Reflectance using Self-supervised Learning and Ray Tracingcited by 48
  11. 2021Neural Video Portrait Relighting in Real-time via Consistency Modelingcited by 46
  12. 2022Face Relighting with Geometrically Consistent Shadowscited by 43
  13. 2020Portrait Shadow Manipulationcited by 42
  14. 2021Towards High Fidelity Face Relighting with Realistic Shadowscited by 42
  15. 2020Learning to Factorize and Relight a Citycited by 36
  16. 2014Multilinear Wavelets: A Statistical Shape Space for Human Facescited by 32
  17. 2021A Light Stage on Every Deskcited by 21
  18. 2020Light Stage Super-Resolution: Continuous High-Frequency Relightingcited by 20
  19. 2020Relightable 3D Head Portraits from a Smartphone Videocited by 8
  20. 2021Deep Portrait Lighting Enhancement with 3D Guidancecited by 2

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2022: 2; 2023: 8; 2024: 9; 2025: 8; 2026: 3.

30 papers.

Sort:

By area:

  1. 2023State of the Art on Diffusion Models for Visual Computingcited by 127
  2. 2024SwitchLight: Co-design of Physics-driven Architecture and Pre-training Framework for Human Portrait Relightingcited by 67
  3. 2024URAvatar: Universal Relightable Gaussian Codec Avatarscited by 43
  4. 2024MeshAvatar: Learning High-quality Triangular Human Avatars from Multi-view Videoscited by 34
  5. 2024Holo-Relighting: Controllable Volumetric Portrait Relighting from a Single Imagecited by 32
  6. 2022HARP: Personalized Hand Reconstruction from a Monocular RGB Videocited by 28
  7. 2025Lux Post Facto: Learning Portrait Performance Relighting with Conditional Video Diffusion and a Hybrid Datasetcited by 28
  8. 2025SynthLight: Portrait Relighting with Diffusion Model by Learning to Re-render Synthetic Facescited by 24
  9. 2022High-Res Facial Appearance Capture from Polarized Smartphone Imagescited by 21
  10. 2023Learning a 3D Morphable Face Reflectance Model from Low-cost Datacited by 20
  11. 2024URHand: Universal Relightable Handscited by 17
  12. 2023LumiGAN: Unconditional Generation of Relightable 3D Human Facescited by 14
18 more
  1. 2023ReliTalk: Relightable Talking Portrait Generation from a Single Videocited by 12
  2. 2023Relightable Neural Actor with Intrinsic Decomposition and Pose Controlcited by 11
  3. 2025TC-Light: Temporally Coherent Generative Rendering for Realistic World Transfercited by 11
  4. 2023High-Quality Facial Geometry and Appearance Capture at Homecited by 9
  5. 2023Personalized Video Relighting With an At-Home Light Stagecited by 7
  6. 2024Text2Relight: Creative Portrait Relighting with Text Guidancecited by 7
  7. 2025IllumiCraft: Unified Geometry and Illumination Diffusion for Controllable Video Generationcited by 7
  8. 2024COMPOSE: Comprehensive Portrait Shadow Editingcited by 6
  9. 2025Parametric Shadow Control for Portrait Generation in Text-to-Image Diffusion Modelscited by 5
  10. 2023SFDM: Robust Decomposition of Geometry and Reflectance for Realistic Face Rendering from Sparse-view Imagescited by 4
  11. 2024LumiSculpt: Enabling Consistent Portrait Lighting in Video Generationcited by 3
  12. 2024Monocular Facial Appearance Capture in the Wildcited by 2
  13. 2025Refracting Reality: Generating Images with Realistic Transparent Objectscited by 2
  14. 2025WildCap: Facial Albedo Capture in the Wild via Hybrid Inverse Renderingcited by 1
  15. 2026KaoLRM: Repurposing Pre-trained Large Reconstruction Models for Parametric 3D Face Reconstructioncited by 1
  16. 2025Using Gaussian Splats to Create High-Fidelity Facial Geometry and Texturecited by 0
  17. 2026RelightAnyone: A Generalized Relightable 3D Gaussian Head Modelcited by 0
  18. 2026Hi-Light: A Path to high-fidelity, high-resolution video relighting with a Novel Evaluation Paradigmcited by 0