# OutCast: Outdoor Single-image Relighting with Cast Shadows

arXiv: https://arxiv.org/abs/2204.09341 · HTML: https://mlsanity.com/paper/2204.09341/

## Key facts

- Cited by 30 papers in the map (data through 14 Oct 2026).
- Builds on 46 of its 94 references that are in the map.
- First posted on arXiv 20 Apr 2022.

## Builds on (most cited first, 30 of 46)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [Implicit Neural Representations with Periodic Activation Functions](https://mlsanity.com/paper/2006.09661/) (2020)
- [pixelNeRF: Neural Radiance Fields from One or Few Images](https://mlsanity.com/paper/2012.02190/) (2020)
- [Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfer](https://mlsanity.com/paper/1907.01341/) (2019)
- [High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANs](https://mlsanity.com/paper/1711.11585/) (2017)
- [Least Squares Generative Adversarial Networks](https://mlsanity.com/paper/1611.04076/) (2016)
- [Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks](https://mlsanity.com/paper/1703.10593/) (2017)
- [NeRV: Neural Reflectance and Visibility Fields for Relighting and View Synthesis](https://mlsanity.com/paper/2012.03927/) (2020)
- [Predicting Depth, Surface Normals and Semantic Labels with a Common Multi-Scale Convolutional Architecture](https://mlsanity.com/paper/1411.4734/) (2014)
- [ARKitScenes: A Diverse Real-World Dataset For 3D Indoor Scene Understanding Using Mobile RGB-D Data](https://mlsanity.com/paper/2111.08897/) (2021)
- [NeRD: Neural Reflectance Decomposition from Image Collections](https://mlsanity.com/paper/2012.03918/) (2020)
- [Learning to Recover 3D Scene Shape from a Single Image](https://mlsanity.com/paper/2012.09365/) (2020)
- [Inverse Rendering for Complex Indoor Scenes: Shape, Spatially-Varying Lighting and SVBRDF from a Single Image](https://mlsanity.com/paper/1905.02722/) (2019)
- [Neural Reflectance Fields for Appearance Acquisition](https://mlsanity.com/paper/2008.03824/) (2020)
- [Escaping Plato's Cave: 3D Shape From Adversarial Rendering](https://mlsanity.com/paper/1811.11606/) (2018)
- [Stacked Conditional Generative Adversarial Networks for Jointly Learning Shadow Detection and Shadow Removal](https://mlsanity.com/paper/1712.02478/) (2017)
- [Single Image Portrait Relighting](https://mlsanity.com/paper/1905.00824/) (2019)
- [NeRFactor: Neural Factorization of Shape and Reflectance Under an Unknown Illumination](https://mlsanity.com/paper/2106.01970/) (2021)
- [Epipolar Transformers](https://mlsanity.com/paper/2005.04551/) (2020)
- [Deep Reflectance Volumes: Relightable Reconstructions from Multi-View Photometric Images](https://mlsanity.com/paper/2007.09892/) (2020)
- [Learning Physics-guided Face Relighting under Directional Light](https://mlsanity.com/paper/1906.03355/) (2019)
- [Deep Parametric Indoor Lighting Estimation](https://mlsanity.com/paper/1910.08812/) (2019)
- [Rendering Synthetic Objects into Legacy Photographs](https://mlsanity.com/paper/1912.11565/) (2019)
- [Neural Face Editing with Intrinsic Image Disentangling](https://mlsanity.com/paper/1704.04131/) (2017)
- [Deep Sky Modeling for Single Image Outdoor Lighting Estimation](https://mlsanity.com/paper/1905.03897/) (2019)
- [DeepLight: Learning Illumination for Unconstrained Mobile Mixed Reality](https://mlsanity.com/paper/1904.01175/) (2019)
- [Inverse Path Tracing for Joint Material and Lighting Estimation](https://mlsanity.com/paper/1903.07145/) (2019)
- [Deep Outdoor Illumination Estimation](https://mlsanity.com/paper/1611.06403/) (2016)

## Built on by (most cited first, 30 of 30)

- [DiLightNet: Fine-grained Lighting Control for Diffusion-based Image Generation](https://mlsanity.com/paper/2402.11929/) (2024)
- [Neural Gaffer: Relighting Any Object via Diffusion](https://mlsanity.com/paper/2406.07520/) (2024)
- [Learning-based Inverse Rendering of Complex Indoor Scenes with Differentiable Monte Carlo Raytracing](https://mlsanity.com/paper/2211.03017/) (2022)
- [LightIt: Illumination Modeling and Control for Diffusion Models](https://mlsanity.com/paper/2403.10615/) (2024)
- [DiffusionRenderer: Neural Inverse and Forward Rendering with Video Diffusion Models](https://mlsanity.com/paper/2501.18590/) (2025)
- [Relighting Neural Radiance Fields with Shadow and Highlight Hints](https://mlsanity.com/paper/2308.13404/) (2023)
- [UniRelight: Learning Joint Decomposition and Synthesis for Video Relighting](https://mlsanity.com/paper/2506.15673/) (2025)
- [LightLab: Controlling Light Sources in Images with Diffusion Models](https://mlsanity.com/paper/2505.09608/) (2025)
- [A Diffusion Approach to Radiance Field Relighting using Multi-Illumination Synthesis](https://mlsanity.com/paper/2409.08947/) (2024)
- [EverLight: Indoor-Outdoor Editable HDR Lighting Estimation](https://mlsanity.com/paper/2304.13207/) (2023)
- [Lux Post Facto: Learning Portrait Performance Relighting with Conditional Video Diffusion and a Hybrid Dataset](https://mlsanity.com/paper/2503.14485/) (2025)
- [GenLit: Reformulating Single-Image Relighting as Video Generation](https://mlsanity.com/paper/2412.11224/) (2024)
- [IntrinsicEdit: Precise generative image manipulation in intrinsic space](https://mlsanity.com/paper/2505.08889/) (2025)
- [Ouroboros: Single-step Diffusion Models for Cycle-consistent Forward and Inverse Rendering](https://mlsanity.com/paper/2508.14461/) (2025)
- [Physically Controllable Relighting of Photographs](https://mlsanity.com/paper/2508.05626/) (2025)
- [GaSLight: Gaussian Splats for Spatially-Varying Lighting in HDR](https://mlsanity.com/paper/2504.10809/) (2025)
- [GaRe: Relightable 3D Gaussian Splatting for Outdoor Scenes from Unconstrained Photo Collections](https://mlsanity.com/paper/2507.20512/) (2025)
- [LuxRemix: Lighting Decomposition and Remixing for Indoor Scenes](https://mlsanity.com/paper/2601.15283/) (2026)
- [MV-CoLight: Efficient Object Compositing with Consistent Lighting and Shadow Generation](https://mlsanity.com/paper/2505.21483/) (2025)
- [PractiLight: Practical Light Control Using Foundational Diffusion Models](https://mlsanity.com/paper/2509.01837/) (2025)
- [MatSwap: Light-aware material transfers in images](https://mlsanity.com/paper/2502.07784/) (2025)
- [PI-Light: Physics-Inspired Diffusion for Full-Image Relighting](https://mlsanity.com/paper/2601.22135/) (2026)
- [All-frequency Full-body Human Image Relighting](https://mlsanity.com/paper/2411.00356/) (2024)
- [A General Albedo Recovery Approach for Aerial Photogrammetric Images through Inverse Rendering](https://mlsanity.com/paper/2409.03032/) (2024)
- [TokenLight: Precise Lighting Control in Images using Attribute Tokens](https://mlsanity.com/paper/2604.15310/) (2026)
- [Towards Physically-Based Sky-Modeling For Image Based Lighting](https://mlsanity.com/paper/2512.15632/) (2025)
- [Joint Shadow Generation and Relighting via Light-Geometry Interaction Maps](https://mlsanity.com/paper/2602.21820/) (2026)
- [SRUG: Shadow-Guided Relightable Urban Scene with Generation Model](https://mlsanity.com/paper/2605.24700/) (2026)
- [BRDFusion: Physics Meets Generation for Urban Scene Inverse Rendering](https://mlsanity.com/paper/2606.17049/) (2026)
- [RelightFormer: Feed-forward Generative Transformer for Multiview Object Relighting](https://mlsanity.com/paper/2609.07414/) (2026)

## Abstract (arXiv)

We propose a relighting method for outdoor images. Our method mainly focuses on predicting cast shadows in arbitrary novel lighting directions from a single image while also accounting for shading and global effects such the sun light color and clouds. Previous solutions for this problem rely on reconstructing occluder geometry, e.g. using multi-view stereo, which requires many images of the scene. Instead, in this work we make use of a noisy off-the-shelf single-image depth map estimation as a source of geometry. Whilst this can be a good guide for some lighting effects, the resulting depth map quality is insufficient for directly ray-tracing the shadows. Addressing this, we propose a learned image space ray-marching layer that converts the approximate depth map into a deep 3D representation that is fused into occlusion queries using a learned traversal. Our proposed method achieves, for the first time, state-of-the-art relighting results, with only a single image as input. For supplementary material visit our project page at: https://dgriffiths.uk/outcast.