# Diffusion-Guided Gaussian Splatting for Large-Scale Unconstrained 3D Reconstruction and Novel View Synthesis

arXiv: https://arxiv.org/abs/2504.01960 · HTML: https://mlsanity.com/paper/2504.01960/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 33 references that are in the map.
- First posted on arXiv 2 Apr 2025.

## Builds on (most cited first, 25 of 25)

- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [3D Gaussian Splatting for Real-Time Radiance Field Rendering](https://mlsanity.com/paper/2308.04079/) (2023)
- [NeRF in the Wild: Neural Radiance Fields for Unconstrained Photo Collections](https://mlsanity.com/paper/2008.02268/) (2020)
- [Scaffold-GS: Structured 3D Gaussians for View-Adaptive Rendering](https://mlsanity.com/paper/2312.00109/) (2023)
- [pixelSplat: 3D Gaussian Splats from Image Pairs for Scalable Generalizable 3D Reconstruction](https://mlsanity.com/paper/2312.12337/) (2023)
- [Mip-Splatting: Alias-free 3D Gaussian Splatting](https://mlsanity.com/paper/2311.16493/) (2023)
- [MVSplat: Efficient 3D Gaussian Splatting from Sparse Multi-View Images](https://mlsanity.com/paper/2403.14627/) (2024)
- [DL3DV-10K: A Large-Scale Scene Dataset for Deep Learning-based 3D Vision](https://mlsanity.com/paper/2312.16256/) (2023)
- [CAT3D: Create Anything in 3D with Multi-View Diffusion Models](https://mlsanity.com/paper/2405.10314/) (2024)
- [ViewCrafter: Taming Video Diffusion Models for High-fidelity Novel View Synthesis](https://mlsanity.com/paper/2409.02048/) (2024)
- [PhysGaussian: Physics-Integrated 3D Gaussians for Generative Dynamics](https://mlsanity.com/paper/2311.12198/) (2023)
- [ReconFusion: 3D Reconstruction with Diffusion Priors](https://mlsanity.com/paper/2312.02981/) (2023)
- [DepthSplat: Connecting Gaussian Splatting and Depth](https://mlsanity.com/paper/2410.13862/) (2024)
- [FSGS: Real-Time Few-shot View Synthesis using Gaussian Splatting](https://mlsanity.com/paper/2312.00451/) (2023)
- [3D Gaussian Splatting as Markov Chain Monte Carlo](https://mlsanity.com/paper/2404.09591/) (2024)
- [WildGaussians: 3D Gaussian Splatting in the Wild](https://mlsanity.com/paper/2407.08447/) (2024)
- [ReconX: Reconstruct Any Scene from Sparse Views with Video Diffusion Model](https://mlsanity.com/paper/2408.16767/) (2024)
- [SparseGS: Sparse View Synthesis using 3D Gaussian Splatting](https://mlsanity.com/paper/2312.00206/) (2023)
- [Gaussian in the Wild: 3D Gaussian Splatting for Unconstrained Image Collections](https://mlsanity.com/paper/2403.15704/) (2024)
- [EscherNet: A Generative Model for Scalable View Synthesis](https://mlsanity.com/paper/2402.03908/) (2024)
- [NeRF On-the-go: Exploiting Uncertainty for Distractor-free NeRFs in the Wild](https://mlsanity.com/paper/2405.18715/) (2024)
- [Spec-Gaussian: Anisotropic View-Dependent Appearance for 3D Gaussian Splatting](https://mlsanity.com/paper/2402.15870/) (2024)
- [SWAG: Splatting in the Wild images with Appearance-conditioned Gaussians](https://mlsanity.com/paper/2403.10427/) (2024)
- [Wild-GS: Real-Time Novel View Synthesis from Unconstrained Photo Collections](https://mlsanity.com/paper/2406.10373/) (2024)
- [BAGS: Blur Agnostic Gaussian Splatting through Multi-Scale Kernel Modeling](https://mlsanity.com/paper/2403.04926/) (2024)

## Built on by (most cited first, 3 of 3)

- [MatSpray: Fusing 2D Material World Knowledge on 3D Geometry](https://mlsanity.com/paper/2512.18314/) (2025)
- [Initialize to Generalize: A Stronger Initialization Pipeline for Sparse-View 3DGS](https://mlsanity.com/paper/2510.17479/) (2025)
- [DF3DV-1K: A Large-Scale Dataset and Benchmark for Distractor-Free Novel View Synthesis](https://mlsanity.com/paper/2604.13416/) (2026)

## Abstract (arXiv)

Recent advancements in 3D Gaussian Splatting (3DGS) and Neural Radiance Fields (NeRF) have achieved impressive results in real-time 3D reconstruction and novel view synthesis. However, these methods struggle in large-scale, unconstrained environments where sparse and uneven input coverage, transient occlusions, appearance variability, and inconsistent camera settings lead to degraded quality. We propose GS-Diff, a novel 3DGS framework guided by a multi-view diffusion model to address these limitations. By generating pseudo-observations conditioned on multi-view inputs, our method transforms under-constrained 3D reconstruction problems into well-posed ones, enabling robust optimization even with sparse data. GS-Diff further integrates several enhancements, including appearance embedding, monocular depth priors, dynamic object modeling, anisotropy regularization, and advanced rasterization techniques, to tackle geometric and photometric challenges in real-world settings. Experiments on four benchmarks demonstrate that GS-Diff consistently outperforms state-of-the-art baselines by significant margins.