mlsanityData to 7 Oct 2026

Learning to Factorize and Relight a City

arXiv 2008.02796 ↗First posted 6 Aug 2020cs.CV cs.GRcited by 36

Not on the map. Why?

Abstract (arXiv)

We propose a learning-based framework for disentangling outdoor scenes into temporally-varying illumination and permanent scene factors. Inspired by the classic intrinsic image decomposition, our learning signal builds upon two insights: 1) combining the disentangled factors should reconstruct the original image, and 2) the permanent factors should stay constant across multiple temporal samples of the same scene. To facilitate training, we assemble a city-scale dataset of outdoor timelapse imagery from Google Street View, where the same locations are captured repeatedly through time. This data represents an unprecedented scale of spatio-temporal outdoor imagery. We show that our learned disentangled factors can be used to manipulate novel images in realistic ways, such as changing lighting effects and scene geometry. Please visit factorize-a-city.github.io for animated results.

Key facts. Cited by 36 papers in the map (data through 7 Oct 2026). First posted on arXiv 6 Aug 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 8; 2022: 8; 2023: 8; 2024: 5; 2025: 4; 2026: 3.

36 papers.

Sort:

By area:

  1. 2021A-SDF: Learning Disentangled Signed Distance Functions for Articulated Shape Representationcited by 113
  2. 2024DiLightNet: Fine-grained Lighting Control for Diffusion-based Image Generationcited by 93
  3. 2024Neural Gaffer: Relighting Any Object via Diffusioncited by 91
  4. 2023Neural Fields meet Explicit Geometric Representation for Inverse Rendering of Urban Scenescited by 89
  5. 2025DiffusionRenderer: Neural Inverse and Forward Rendering with Video Diffusion Modelscited by 55
  6. 2023Stanford-ORB: A Real-World 3D Object Inverse Rendering Benchmarkcited by 51
  7. 2022IntrinsicNeRF: Learning Intrinsic Neural Radiance Fields for Editable Novel View Synthesiscited by 50
  8. 2025UniRelight: Learning Joint Decomposition and Synthesis for Video Relightingcited by 38
  9. 2024A Diffusion Approach to Radiance Field Relighting using Multi-Illumination Synthesiscited by 37
  10. 2024LumiNet: Latent Intrinsics Meets Diffusion Models for Indoor Scene Relightingcited by 37
  11. 2021Video Autoencoder: self-supervised disentanglement of static 3D structure and motioncited by 34
  12. 2022De-rendering 3D Objects in the Wildcited by 31
24 more
  1. 2022SunStage: Portrait Reconstruction and Relighting using the Sun as a Light Stagecited by 30
  2. 2022OutCast: Outdoor Single-image Relighting with Cast Shadowscited by 30
  3. 2021De-rendering the World's Revolutionary Artefactscited by 28
  4. 2022Estimating Reflectance Layer from A Single Image: Integrating Reflectance Guidance and Shadow/Specular Aware Learningcited by 18
  5. 2024Latent Intrinsics Emerge from Training to Relightcited by 17
  6. 2023NeRRF: 3D Reconstruction and View Synthesis for Transparent and Specular Objects with Neural Refractive-Reflective Fieldscited by 16
  7. 2022MonoNeRF: Learning Generalizable NeRFs from Monocular Videos without Camera Posecited by 10
  8. 2023Neural Scene Chronologycited by 10
  9. 2022What's in a Decade? Transforming Faces Through Timecited by 9
  10. 2023TUVF: Learning Generalizable Texture UV Radiance Fieldscited by 9
  11. 2023Inverse Rendering of Translucent Objects using Physical and Neural Rendererscited by 9
  12. 2026Generative World Renderercited by 8
  13. 2023Fast Sun-aligned Outdoor Scene Relighting based on TensoRFcited by 7
  14. 2025PractiLight: Practical Light Control Using Foundational Diffusion Modelscited by 7
  15. 2021Repopulating Street Scenescited by 6
  16. 2021DeRenderNet: Intrinsic Image Decomposition of Urban Scenes with Shape-(In)dependent Shading Renderingcited by 6
  17. 2025GPS as a Control Signal for Image Generationcited by 5
  18. 2021SILT: Self-supervised Lighting Transfer Using Implicit Image Decompositioncited by 4
  19. 2021Controllable Data Augmentation Through Deep Relightingcited by 4
  20. 2022Time-of-Day Neural Style Transfer for Architectural Photographscited by 3
  21. 2026Time-Archival Camera Virtualization for Sports and Visual Performancescited by 1
  22. 2021Exploring Biases and Prejudice of Facial Synthesis via Semantic Latent Spacecited by 0
  23. 2023Neural Implicit Field Editing Considering Object-environment Interactioncited by 0
  24. 2026Real-time Neural Six-way Lightmapscited by 0