# WE-GS: An In-the-wild Efficient 3D Gaussian Representation for Unconstrained Photo Collections

arXiv: https://arxiv.org/abs/2406.02407 · HTML: https://mlsanity.com/paper/2406.02407/

## Key facts

- Cited by 33 papers in the map (data through 14 Oct 2026).
- Builds on 33 of its 47 references that are in the map.
- First posted on arXiv 4 Jun 2024.

## Builds on (most cited first, 30 of 33)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [3D Gaussian Splatting for Real-Time Radiance Field Rendering](https://mlsanity.com/paper/2308.04079/) (2023)
- [PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation](https://mlsanity.com/paper/1612.00593/) (2016)
- [Instant Neural Graphics Primitives with a Multiresolution Hash Encoding](https://mlsanity.com/paper/2201.05989/) (2022)
- [CBAM: Convolutional Block Attention Module](https://mlsanity.com/paper/1807.06521/) (2018)
- [Mip-NeRF 360: Unbounded Anti-Aliased Neural Radiance Fields](https://mlsanity.com/paper/2111.12077/) (2021)
- [TensoRF: Tensorial Radiance Fields](https://mlsanity.com/paper/2203.09517/) (2022)
- [NeRF in the Wild: Neural Radiance Fields for Unconstrained Photo Collections](https://mlsanity.com/paper/2008.02268/) (2020)
- [DreamGaussian: Generative Gaussian Splatting for Efficient 3D Content Creation](https://mlsanity.com/paper/2309.16653/) (2023)
- [K-Planes: Explicit Radiance Fields in Space, Time, and Appearance](https://mlsanity.com/paper/2301.10241/) (2023)
- [Scaffold-GS: Structured 3D Gaussians for View-Adaptive Rendering](https://mlsanity.com/paper/2312.00109/) (2023)
- [Mip-Splatting: Alias-free 3D Gaussian Splatting](https://mlsanity.com/paper/2311.16493/) (2023)
- [AtlasNet: A Papier-Mach'e Approach to Learning 3D Surface Generation](https://mlsanity.com/paper/1802.05384/) (2018)
- [PhysGaussian: Physics-Integrated 3D Gaussians for Generative Dynamics](https://mlsanity.com/paper/2311.12198/) (2023)
- [Compact 3D Gaussian Representation for Radiance Field](https://mlsanity.com/paper/2311.13681/) (2023)
- [MobileNeRF: Exploiting the Polygon Rasterization Pipeline for Efficient Neural Field Rendering on Mobile Architectures](https://mlsanity.com/paper/2208.00277/) (2022)
- [GS-SLAM: Dense Visual SLAM with 3D Gaussian Splatting](https://mlsanity.com/paper/2311.11700/) (2023)
- [Volumetric and Multi-View CNNs for Object Classification on 3D Data](https://mlsanity.com/paper/1604.03265/) (2016)
- [Image Matching across Wide Baselines: From Paper to Practice](https://mlsanity.com/paper/2003.01587/) (2020)
- [MERF: Memory-Efficient Radiance Fields for Real-time View Synthesis in Unbounded Scenes](https://mlsanity.com/paper/2302.12249/) (2023)
- [NeRF-Editing: Geometry Editing of Neural Radiance Fields](https://mlsanity.com/paper/2205.04978/) (2022)
- [Language Embedded 3D Gaussians for Open-Vocabulary Scene Understanding](https://mlsanity.com/paper/2311.18482/) (2023)
- [Tri-MipRF: Tri-Mip Representation for Efficient Anti-Aliasing Neural Radiance Fields](https://mlsanity.com/paper/2307.11335/) (2023)
- [Hallucinated Neural Radiance Fields in the Wild](https://mlsanity.com/paper/2111.15246/) (2021)
- [Gaussian in the Wild: 3D Gaussian Splatting for Unconstrained Image Collections](https://mlsanity.com/paper/2403.15704/) (2024)
- [VSGNet: Spatial Attention Network for Detecting Human Object Interactions Using Graph Convolutions](https://mlsanity.com/paper/2003.05541/) (2020)
- [NeRF for Outdoor Scene Relighting](https://mlsanity.com/paper/2112.05140/) (2021)
- [SA-UNet: Spatial Attention U-Net for Retinal Vessel Segmentation](https://mlsanity.com/paper/2004.03696/) (2020)
- [SWAG: Splatting in the Wild images with Appearance-conditioned Gaussians](https://mlsanity.com/paper/2403.10427/) (2024)

## Built on by (most cited first, 30 of 33)

- [WildGaussians: 3D Gaussian Splatting in the Wild](https://mlsanity.com/paper/2407.08447/) (2024)
- [SpotlessSplats: Ignoring Distractors in 3D Gaussian Splatting](https://mlsanity.com/paper/2406.20055/) (2024)
- [Director3D: Real-world Camera Trajectory and 3D Scene Generation from Text](https://mlsanity.com/paper/2406.17601/) (2024)
- [NerfBaselines: Consistent and Reproducible Evaluation of Novel View Synthesis Methods](https://mlsanity.com/paper/2406.17345/) (2024)
- [Drag Your Gaussian: Effective Drag-Based Editing with Score Distillation for 3D Gaussian Splatting](https://mlsanity.com/paper/2501.18672/) (2025)
- [LumiGauss: Relightable Gaussian Splatting in the Wild](https://mlsanity.com/paper/2408.04474/) (2024)
- [DeRainGS: Gaussian Splatting for Enhanced Scene Reconstruction in Rainy Environments](https://mlsanity.com/paper/2408.11540/) (2024)
- [Skyfall-GS: Synthesizing Immersive 3D Urban Scenes from Satellite Imagery](https://mlsanity.com/paper/2510.15869/) (2025)
- [HybridGS: Decoupling Transients and Statics with 2D and 3D Gaussian Splatting](https://mlsanity.com/paper/2412.03844/) (2024)
- [Dense-SfM: Structure from Motion with Dense Consistent Matching](https://mlsanity.com/paper/2501.14277/) (2025)
- [RobustSplat: Decoupling Densification and Dynamics for Transient-Free 3DGS](https://mlsanity.com/paper/2506.02751/) (2025)
- [Micro-macro Wavelet-based Gaussian Splatting for 3D Reconstruction from Unconstrained Images](https://mlsanity.com/paper/2501.14231/) (2025)
- [Evolving High-Quality Rendering and Reconstruction in a Unified Framework with Contribution-Adaptive Regularization](https://mlsanity.com/paper/2503.00881/) (2025)
- [Training-Free Hierarchical Scene Understanding for Gaussian Splatting with Superpoint Graphs](https://mlsanity.com/paper/2504.13153/) (2025)
- [NexusSplats: Efficient 3D Gaussian Splatting in the Wild](https://mlsanity.com/paper/2411.14514/) (2024)
- [Seg-Wild: Interactive Segmentation based on 3D Gaussian Splatting for Unconstrained Image Collections](https://mlsanity.com/paper/2507.07395/) (2025)
- [XSpecMesh: Quality-Preserving Auto-Regressive Mesh Generation Acceleration via Multi-Head Speculative Decoding](https://mlsanity.com/paper/2507.23777/) (2025)
- [SimVS: Simulating World Inconsistencies for Robust View Synthesis](https://mlsanity.com/paper/2412.07696/) (2024)
- [CHROMA: Consistent Harmonization of Multi-View Appearance via Bilateral Grid Prediction](https://mlsanity.com/paper/2507.15748/) (2025)
- [WildRayZer: Self-supervised Large View Synthesis in Dynamic Environments](https://mlsanity.com/paper/2601.10716/) (2026)
- [SparseGS-W: Sparse-View 3D Gaussian Splatting in the Wild with Generative Priors](https://mlsanity.com/paper/2503.19452/) (2025)
- [Micro-macro Gaussian Splatting with Enhanced Scalability for Unconstrained Scene Reconstruction](https://mlsanity.com/paper/2506.13516/) (2025)
- [Text-driven 3D Human Generation via Contrastive Preference Optimization](https://mlsanity.com/paper/2502.08977/) (2025)
- [Taking Language Embedded 3D Gaussian Splatting into the Wild](https://mlsanity.com/paper/2507.19830/) (2025)
- [OCSplats: Observation Completeness Quantification and Label Noise Separation in 3DGS](https://mlsanity.com/paper/2508.01239/) (2025)
- [ChronoGS: Disentangling Invariants and Changes in Multi-Period Scenes](https://mlsanity.com/paper/2511.18794/) (2025)
- [Generalizable Sparse-View 3D Reconstruction from Unconstrained Images](https://mlsanity.com/paper/2604.28193/) (2026)
- [WilLaGS: Latent-Conditional 3D Appearance Fields for Robust Gaussian Splatting In-the-Wild](https://mlsanity.com/paper/2608.28240/) (2026)
- [ArchGPT: Understanding the World's Architectures with Large Multimodal Models](https://mlsanity.com/paper/2509.20858/) (2025)
- [WildGHand: Learning Anti-Perturbation Gaussian Hand Avatars from Monocular In-the-Wild Videos](https://mlsanity.com/paper/2602.20556/) (2026)

## Abstract (arXiv)

Novel View Synthesis (NVS) from unconstrained photo collections is challenging in computer graphics. Recently, 3D Gaussian Splatting (3DGS) has shown promise for photorealistic and real-time NVS of static scenes. Building on 3DGS, we propose an efficient point-based differentiable rendering framework for scene reconstruction from photo collections. Our key innovation is a residual-based spherical harmonic coefficients transfer module that adapts 3DGS to varying lighting conditions and photometric post-processing. This lightweight module can be pre-computed and ensures efficient gradient propagation from rendered images to 3D Gaussian attributes. Additionally, we observe that the appearance encoder and the transient mask predictor, the two most critical parts of NVS from unconstrained photo collections, can be mutually beneficial. We introduce a plug-and-play lightweight spatial attention module to simultaneously predict transient occluders and latent appearance representation for each image. After training and preprocessing, our method aligns with the standard 3DGS format and rendering pipeline, facilitating seamlessly integration into various 3DGS applications. Extensive experiments on diverse datasets show our approach outperforms existing approaches on the rendering quality of novel view and appearance synthesis with high converge and rendering speed.