# What Builds the Scene? Luminance Dominates Geometry Formation in 3D Gaussian Splatting

arXiv: https://arxiv.org/abs/2610.00749 · HTML: https://mlsanity.com/paper/2610.00749/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 26 of its 37 references that are in the map.
- First posted on arXiv 30 Sep 2026.

## Builds on (most cited first, 26 of 26)

- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [3D Gaussian Splatting for Real-Time Radiance Field Rendering](https://mlsanity.com/paper/2308.04079/) (2023)
- [Mip-NeRF 360: Unbounded Anti-Aliased Neural Radiance Fields](https://mlsanity.com/paper/2111.12077/) (2021)
- [Ref-NeRF: Structured View-Dependent Appearance for Neural Radiance Fields](https://mlsanity.com/paper/2112.03907/) (2021)
- [Compact 3D Gaussian Representation for Radiance Field](https://mlsanity.com/paper/2311.13681/) (2023)
- [Compressed 3D Gaussian Splatting for Accelerated Novel View Synthesis](https://mlsanity.com/paper/2401.02436/) (2023)
- [Taming 3DGS: High-Quality Radiance Fields with Limited Resources](https://mlsanity.com/paper/2406.15643/) (2024)
- [Reducing the Memory Footprint of 3D Gaussian Splatting](https://mlsanity.com/paper/2406.17074/) (2024)
- [Revising Densification in Gaussian Splatting](https://mlsanity.com/paper/2404.06109/) (2024)
- [FreGS: 3D Gaussian Splatting with Progressive Frequency Regularization](https://mlsanity.com/paper/2403.06908/) (2024)
- [AbsGS: Recovering Fine Details for 3D Gaussian Splatting](https://mlsanity.com/paper/2404.10484/) (2024)
- [FastGS: Training 3D Gaussian Splatting in 100 Seconds](https://mlsanity.com/paper/2511.04283/) (2025)
- [Luminance-GS: Adapting 3D Gaussian Splatting to Challenging Lighting Conditions with View-Adaptive Curve Adjustment](https://mlsanity.com/paper/2504.01503/) (2025)
- [Colorizing Monochromatic Radiance Fields](https://mlsanity.com/paper/2402.12184/) (2024)
- [EntropyGS: An Efficient Entropy Coding on 3D Gaussian Splatting](https://mlsanity.com/paper/2508.10227/) (2025)
- [From Coarse to Fine: Learnable Discrete Wavelet Transforms for Efficient 3D Gaussian Splatting](https://mlsanity.com/paper/2506.23042/) (2025)
- [ChromaDistill: Colorizing Monochrome Radiance Fields with Knowledge Distillation](https://mlsanity.com/paper/2309.07668/) (2023)
- [Optimized 3D Gaussian Splatting using Coarse-to-Fine Image Frequency Modulation](https://mlsanity.com/paper/2503.14475/) (2025)
- [Structured Image-based Coding for Efficient Gaussian Splatting Compression](https://mlsanity.com/paper/2601.14510/) (2026)
- [Color3D: Controllable and Consistent 3D Colorization with Personalized Colorizer](https://mlsanity.com/paper/2510.10152/) (2025)
- [HiGS: A Hierarchical Rendering Architecture for Real-Time 3D Gaussian Splatting](https://mlsanity.com/paper/2606.00352/) (2026)
- [High Dynamic Range 3D Gaussian Splatting via Luminance-Chromaticity Decomposition](https://mlsanity.com/paper/2511.12895/) (2025)
- [Thinking Like Van Gogh: Structure-Aware Style Transfer via Flow-Guided 3D Gaussian Splatting](https://mlsanity.com/paper/2601.10075/) (2026)
- [PDF-GS: Progressive Distractor Filtering for Robust 3D Gaussian Splatting](https://mlsanity.com/paper/2604.12580/) (2026)
- [When 3D Gaussian Splatting Recovers Real Surfaces](https://mlsanity.com/paper/2608.30054/) (2026)
- [Reparametrizing 3D Gaussian Splatting for Real-Time Palette-based Color and Luminance Editing](https://mlsanity.com/paper/2609.03897/) (2026)

## Abstract (arXiv)

Standard 3D Gaussian Splatting (3DGS) learns geometry and appearance jointly from RGB supervision, making it difficult to isolate how luminance and chroma contribute to the learned representation. We study this by training models under different channel supervision, freezing their non-appearance parameters (position, scale, rotation, and opacity), and re-estimating appearance with the same solver before comparing held-out reconstruction. Across eleven benchmark scenes with four independent runs each, geometry learned from luminance alone supports held-out reconstruction 0.085 dB below RGB-trained geometry on average. If chroma is deleted from a trained model, a sufficiently expressive solver can re-fit it on the frozen geometry to the original quality or slightly better. Higher-order spherical harmonics contribute much more reconstruction quality to luminance than to chroma, improving PSNR by 1.44 dB versus 0.19 dB on average, although on mirror-like surfaces hue does still change with viewpoint. The luminance advantage is even larger when geometry is being formed. Chroma-only supervision produces geometry 3.9-5.5 dB worse than luminance-only supervision after the same appearance solve; densification explains part of this gap. Overall, geometry formation in standard 3DGS is strongly luminance-dominated but not luminance-exclusive, and much of the chromatic appearance can be recovered after spatial support has formed.