# DeferredGS: Decoupled and Editable Gaussian Splatting with Deferred Shading

arXiv: https://arxiv.org/abs/2404.09412 · HTML: https://mlsanity.com/paper/2404.09412/

## Key facts

- Cited by 41 papers in the map (data through 14 Oct 2026).
- Builds on 51 of its 76 references that are in the map.
- First posted on arXiv 15 Apr 2024.

## Builds on (most cited first, 30 of 51)

- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [3D Gaussian Splatting for Real-Time Radiance Field Rendering](https://mlsanity.com/paper/2308.04079/) (2023)
- [Adding Conditional Control to Text-to-Image Diffusion Models](https://mlsanity.com/paper/2302.05543/) (2023)
- [Instant Neural Graphics Primitives with a Multiresolution Hash Encoding](https://mlsanity.com/paper/2201.05989/) (2022)
- [DreamFusion: Text-to-3D using 2D Diffusion](https://mlsanity.com/paper/2209.14988/) (2022)
- [DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation](https://mlsanity.com/paper/1901.05103/) (2019)
- [Occupancy Networks: Learning 3D Reconstruction in Function Space](https://mlsanity.com/paper/1812.03828/) (2018)
- [NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstruction](https://mlsanity.com/paper/2106.10689/) (2021)
- [TensoRF: Tensorial Radiance Fields](https://mlsanity.com/paper/2203.09517/) (2022)
- [2D Gaussian Splatting for Geometrically Accurate Radiance Fields](https://mlsanity.com/paper/2403.17888/) (2024)
- [Learning Implicit Fields for Generative Shape Modeling](https://mlsanity.com/paper/1812.02822/) (2018)
- [Volume Rendering of Neural Implicit Surfaces](https://mlsanity.com/paper/2106.12052/) (2021)
- [Direct Voxel Grid Optimization: Super-fast Convergence for Radiance Fields Reconstruction](https://mlsanity.com/paper/2111.11215/) (2021)
- [Neural Sparse Voxel Fields](https://mlsanity.com/paper/2007.11571/) (2020)
- [Differentiable Volumetric Rendering: Learning Implicit 3D Representations without 3D Supervision](https://mlsanity.com/paper/1912.07372/) (2019)
- [UNISURF: Unifying Neural Implicit Surfaces and Radiance Fields for Multi-View Reconstruction](https://mlsanity.com/paper/2104.10078/) (2021)
- [SuGaR: Surface-Aligned Gaussian Splatting for Efficient 3D Mesh Reconstruction and High-Quality Mesh Rendering](https://mlsanity.com/paper/2311.12775/) (2023)
- [Ref-NeRF: Structured View-Dependent Appearance for Neural Radiance Fields](https://mlsanity.com/paper/2112.03907/) (2021)
- [NeRV: Neural Reflectance and Visibility Fields for Relighting and View Synthesis](https://mlsanity.com/paper/2012.03927/) (2020)
- [Instruct-NeRF2NeRF: Editing 3D Scenes with Instructions](https://mlsanity.com/paper/2303.12789/) (2023)
- [Deep Marching Tetrahedra: a Hybrid Representation for High-Resolution 3D Shape Synthesis](https://mlsanity.com/paper/2111.04276/) (2021)
- [NeRD: Neural Reflectance Decomposition from Image Collections](https://mlsanity.com/paper/2012.03918/) (2020)
- [Extracting Triangular 3D Models, Materials, and Lighting From Images](https://mlsanity.com/paper/2111.12503/) (2021)
- [PhySG: Inverse Rendering with Spherical Gaussians for Physics-based Material Editing and Relighting](https://mlsanity.com/paper/2104.00674/) (2021)
- [GaussianEditor: Swift and Controllable 3D Editing with Gaussian Splatting](https://mlsanity.com/paper/2311.14521/) (2023)
- [MobileNeRF: Exploiting the Polygon Rasterization Pipeline for Efficient Neural Field Rendering on Mobile Architectures](https://mlsanity.com/paper/2208.00277/) (2022)
- [GaussianShader: 3D Gaussian Splatting with Shading Functions for Reflective Surfaces](https://mlsanity.com/paper/2311.17977/) (2023)
- [GS-IR: 3D Gaussian Splatting for Inverse Rendering](https://mlsanity.com/paper/2311.16473/) (2023)
- [Editing Conditional Radiance Fields](https://mlsanity.com/paper/2105.06466/) (2021)

## Built on by (most cited first, 30 of 41)

- [GI-GS: Global Illumination Decomposition on Gaussian Splatting for Inverse Rendering](https://mlsanity.com/paper/2410.02619/) (2024)
- [Ref-GS: Directional Factorization for 2D Gaussian Splatting](https://mlsanity.com/paper/2412.00905/) (2024)
- [HRAvatar: High-Quality and Relightable Gaussian Head Avatar](https://mlsanity.com/paper/2503.08224/) (2025)
- [DecoupledGaussian: Object-Scene Decoupling for Physics-Based Interaction](https://mlsanity.com/paper/2503.05484/) (2025)
- [Ouroboros: Single-step Diffusion Models for Cycle-consistent Forward and Inverse Rendering](https://mlsanity.com/paper/2508.14461/) (2025)
- [Neural Shell Texture Splatting: More Details and Fewer Primitives](https://mlsanity.com/paper/2507.20200/) (2025)
- [GlossyGS: Inverse Rendering of Glossy Objects with 3D Gaussian Splatting](https://mlsanity.com/paper/2410.13349/) (2024)
- [Gaussian Splatting with Discretized SDF for Relightable Assets](https://mlsanity.com/paper/2507.15629/) (2025)
- [RTR-GS: 3D Gaussian Splatting for Inverse Rendering with Radiance Transfer and Reflection](https://mlsanity.com/paper/2507.07733/) (2025)
- [3D Gaussian Inverse Rendering with Approximated Global Illumination](https://mlsanity.com/paper/2504.01358/) (2025)
- [VGD: Visual Geometry Gaussian Splatting for Feed-Forward Surround-view Driving Reconstruction](https://mlsanity.com/paper/2510.19578/) (2025)
- [Normal-GS: 3D Gaussian Splatting with Normal-Involved Rendering](https://mlsanity.com/paper/2410.20593/) (2024)
- [Digital Twin Generation from Visual Data: A Survey](https://mlsanity.com/paper/2504.13159/) (2025)
- [HumanOLAT: A Large-Scale Dataset for Full-Body Human Relighting and Novel-View Synthesis](https://mlsanity.com/paper/2508.09137/) (2025)
- [Interactive Rendering of Relightable and Animatable Gaussian Avatars](https://mlsanity.com/paper/2407.10707/) (2024)
- [MVInverse: Feed-forward Multi-view Inverse Rendering in Seconds](https://mlsanity.com/paper/2512.21003/) (2025)
- [Progressive Radiance Distillation for Inverse Rendering with Gaussian Splatting](https://mlsanity.com/paper/2408.07595/) (2024)
- [MetaGS: A Meta-Learned Gaussian-Phong Model for Out-of-Distribution 3D Scene Relighting](https://mlsanity.com/paper/2405.20791/) (2024)
- [PR-ENDO: Physically Based Relightable Gaussian Splatting for Endoscopy](https://mlsanity.com/paper/2411.12510/) (2024)
- [Generalizable and Relightable Gaussian Splatting for Human Novel View Synthesis](https://mlsanity.com/paper/2505.21502/) (2025)
- [RISE-SDF: a Relightable Information-Shared Signed Distance Field for Glossy Object Inverse Rendering](https://mlsanity.com/paper/2409.20140/) (2024)
- [RGS-DR: Deferred Reflections and Residual Shading in 2D Gaussian Splatting](https://mlsanity.com/paper/2504.18468/) (2025)
- [Monocular Facial Appearance Capture in the Wild](https://mlsanity.com/paper/2412.12765/) (2024)
- [DNRSelect: Active Best View Selection for Deferred Neural Rendering](https://mlsanity.com/paper/2501.12150/) (2025)
- [Micro-macro Gaussian Splatting with Enhanced Scalability for Unconstrained Scene Reconstruction](https://mlsanity.com/paper/2506.13516/) (2025)
- [ROSGS: Relightable Outdoor Scenes With Gaussian Splatting](https://mlsanity.com/paper/2509.11275/) (2025)
- [UniGS: Unified Geometry-Aware Gaussian Splatting for Multimodal Rendering](https://mlsanity.com/paper/2510.12174/) (2025)
- [RT-Splatting: Joint Reflection-Transmission Modeling with Gaussian Splatting](https://mlsanity.com/paper/2605.18263/) (2026)
- [SPARE-GS: Structural Parsimony and Resource Efficiency for 3D Gaussian Splatting](https://mlsanity.com/paper/2607.16624/) (2026)
- [PS-GS: Gaussian Splatting for Multi-View Photometric Stereo](https://mlsanity.com/paper/2507.18231/) (2025)

## Abstract (arXiv)

Reconstructing and editing 3D objects and scenes both play crucial roles in computer graphics and computer vision. Neural radiance fields (NeRFs) can achieve realistic reconstruction and editing results but suffer from inefficiency in rendering. Gaussian splatting significantly accelerates rendering by rasterizing Gaussian ellipsoids. However, Gaussian splatting utilizes a single Spherical Harmonic (SH) function to model both texture and lighting, limiting independent editing capabilities of these components. Recently, attempts have been made to decouple texture and lighting with the Gaussian splatting representation but may fail to produce plausible geometry and decomposition results on reflective scenes. Additionally, the forward shading technique they employ introduces noticeable blending artifacts during relighting, as the geometry attributes of Gaussians are optimized under the original illumination and may not be suitable for novel lighting conditions. To address these issues, we introduce DeferredGS, a method for decoupling and editing the Gaussian splatting representation using deferred shading. To achieve successful decoupling, we model the illumination with a learnable environment map and define additional attributes such as texture parameters and normal direction on Gaussians, where the normal is distilled from a jointly trained signed distance function. More importantly, we apply deferred shading, resulting in more realistic relighting effects compared to previous methods. Both qualitative and quantitative experiments demonstrate the superior performance of DeferredGS in novel view synthesis and editing tasks.