# CoherentGS: Sparse Novel View Synthesis with Coherent 3D Gaussians

arXiv: https://arxiv.org/abs/2403.19495 · HTML: https://mlsanity.com/paper/2403.19495/

## Key facts

- Cited by 62 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 Mar 2024.

## Built on by (most cited first, 30 of 62)

- [3D Gaussian Splatting: Survey, Technologies, Challenges, and Opportunities](https://mlsanity.com/paper/2407.17418/) (2024)
- [DropGaussian: Structural Regularization for Sparse-view Gaussian Splatting](https://mlsanity.com/paper/2504.00773/) (2025)
- [PhysMotion: Physics-Grounded Dynamics From a Single Image](https://mlsanity.com/paper/2411.17189/) (2024)
- [Binocular-Guided 3D Gaussian Splatting with View Consistency for Sparse View Synthesis](https://mlsanity.com/paper/2410.18822/) (2024)
- [Taming Video Diffusion Prior with Scene-Grounding Guidance for 3D Gaussian Splatting from Sparse Inputs](https://mlsanity.com/paper/2503.05082/) (2025)
- [MAtCha Gaussians: Atlas of Charts for High-Quality Geometry and Photorealism From Sparse Views](https://mlsanity.com/paper/2412.06767/) (2024)
- [FewViewGS: Gaussian Splatting with Few View Matching and Multi-stage Training](https://mlsanity.com/paper/2411.02229/) (2024)
- [GGS: Generalizable Gaussian Splatting for Lane Switching in Autonomous Driving](https://mlsanity.com/paper/2409.02382/) (2024)
- [RI3D: Few-Shot Gaussian Splatting With Repair and Inpainting Diffusion Priors](https://mlsanity.com/paper/2503.10860/) (2025)
- [Self-Ensembling Gaussian Splatting for Few-Shot Novel View Synthesis](https://mlsanity.com/paper/2411.00144/) (2024)
- [Quantifying and Alleviating Co-Adaptation in Sparse-View 3D Gaussian Splatting](https://mlsanity.com/paper/2508.12720/) (2025)
- [SolidGS: Consolidating Gaussian Surfel Splatting for Sparse-View Surface Reconstruction](https://mlsanity.com/paper/2412.15400/) (2024)
- [Next Best Sense: Guiding Vision and Touch with FisherRF for 3D Gaussian Splatting](https://mlsanity.com/paper/2410.04680/) (2024)
- [OmniSplat: Taming Feed-Forward 3D Gaussian Splatting for Omnidirectional Images with Editable Capabilities](https://mlsanity.com/paper/2412.16604/) (2024)
- [CoMapGS: Covisibility Map-based Gaussian Splatting for Sparse Novel View Synthesis](https://mlsanity.com/paper/2503.20998/) (2025)
- [G4Splat: Geometry-Guided Gaussian Splatting with Generative Prior](https://mlsanity.com/paper/2510.12099/) (2025)
- [Aquatic-GS: A Hybrid 3D Representation for Underwater Scenes](https://mlsanity.com/paper/2411.00239/) (2024)
- [SPARS3R: Semantic Prior Alignment and Regularization for Sparse 3D Reconstruction](https://mlsanity.com/paper/2411.12592/) (2024)
- [Turbo-GS: Accelerating 3D Gaussian Fitting for High-Quality Radiance Fields](https://mlsanity.com/paper/2412.13547/) (2024)
- [AD-GS: Alternating Densification for Sparse-Input 3D Gaussian Splatting](https://mlsanity.com/paper/2509.11003/) (2025)
- [Digital Twin Generation from Visual Data: A Survey](https://mlsanity.com/paper/2504.13159/) (2025)
- [AIM 2024 Sparse Neural Rendering Challenge: Dataset and Benchmark](https://mlsanity.com/paper/2409.15041/) (2024)
- [MCGS: Multiview Consistency Enhancement for Sparse-View 3D Gaussian Radiance Fields](https://mlsanity.com/paper/2410.11394/) (2024)
- [Sparse4DGS: 4D Gaussian Splatting for Sparse-Frame Dynamic Scene Reconstruction](https://mlsanity.com/paper/2511.07122/) (2025)
- [Novel View Synthesis from A Few Glimpses via Test-Time Natural Video Completion](https://mlsanity.com/paper/2511.17932/) (2025)
- [GaMO: Geometry-aware Multi-view Diffusion Outpainting for Sparse-View 3D Reconstruction](https://mlsanity.com/paper/2512.25073/) (2025)
- [SparseGS-W: Sparse-View 3D Gaussian Splatting in the Wild with Generative Priors](https://mlsanity.com/paper/2503.19452/) (2025)
- [VGNC: Reducing the Overfitting of Sparse-view 3DGS via Validation-guided Gaussian Number Control](https://mlsanity.com/paper/2504.14548/) (2025)
- [JointSplat: Probabilistic Joint Flow-Depth Optimization for Sparse-View Gaussian Splatting](https://mlsanity.com/paper/2506.03872/) (2025)
- [Geometry-Consistent 4D Gaussian Splatting for Sparse-Input Dynamic View Synthesis](https://mlsanity.com/paper/2511.23044/) (2025)

## Abstract (arXiv)

The field of 3D reconstruction from images has rapidly evolved in the past few years, first with the introduction of Neural Radiance Field (NeRF) and more recently with 3D Gaussian Splatting (3DGS). The latter provides a significant edge over NeRF in terms of the training and inference speed, as well as the reconstruction quality. Although 3DGS works well for dense input images, the unstructured point-cloud like representation quickly overfits to the more challenging setup of extremely sparse input images (e.g., 3 images), creating a representation that appears as a jumble of needles from novel views. To address this issue, we propose regularized optimization and depth-based initialization. Our key idea is to introduce a structured Gaussian representation that can be controlled in 2D image space. We then constraint the Gaussians, in particular their position, and prevent them from moving independently during optimization. Specifically, we introduce single and multiview constraints through an implicit convolutional decoder and a total variation loss, respectively. With the coherency introduced to the Gaussians, we further constrain the optimization through a flow-based loss function. To support our regularized optimization, we propose an approach to initialize the Gaussians using monocular depth estimates at each input view. We demonstrate significant improvements compared to the state-of-the-art sparse-view NeRF-based approaches on a variety of scenes.