Enhancing Novel View Synthesis from extremely sparse views with SfM-free 3D Gaussian Splatting Framework
Not on the map. Why?
Abstract (arXiv)
3D Gaussian Splatting (3DGS) has demonstrated remarkable real-time performance in novel view synthesis, yet its effectiveness relies heavily on dense multi-view inputs with precisely known camera poses, which are rarely available in real-world scenarios. When input views become extremely sparse, the Structure-from-Motion (SfM) method that 3DGS depends on for initialization fails to accurately reconstruct the 3D geometric structures of scenes, resulting in degraded rendering quality. In this paper, we propose a novel SfM-free 3DGS-based method that jointly estimates camera poses and reconstructs 3D scenes from extremely sparse-view inputs. Specifically, instead of SfM, we propose a dense stereo module to progressively estimates camera pose information and reconstructs a global dense point cloud for initialization. To address the inherent problem of information scarcity in extremely sparse-view settings, we propose a coherent view interpolation module that interpolates camera poses based on training view pairs and generates viewpoint-consistent content as additional supervision signals for training. Furthermore, we introduce multi-scale Laplacian consistent regularization and adaptive spatial-aware multi-scale geometry regularization to enhance the quality of geometrical structures and rendered content. Experiments show that our method significantly outperforms other state-of-the-art 3DGS-based approaches, achieving a remarkable 2.75dB improvement in PSNR under extremely sparse-view conditions (using only 2 training views). The images synthesized by our method exhibit minimal distortion while preserving rich high-frequency details, resulting in superior visual quality compared to existing techniques.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 53 of its 57 references that are in the map. First posted on arXiv 21 Aug 2025.
Builds on
53 of 57 references.
References · 53
- 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,177
- 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,226
- 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,025
- 2021Mip-NeRF 360: Unbounded Anti-Aliased Neural Radiance Fieldscited by 2,032
- 2021Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fieldscited by 1,964
- 2023DUSt3R: Geometric 3D Vision Made Easycited by 1,567
- 2023Zero-1-to-3: Zero-shot One Image to 3D Objectcited by 1,489
- 2022Magic3D: High-Resolution Text-to-3D Content Creationcited by 1,290
- 2023ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillationcited by 1,219
- 2022Diffusion Models in Vision: A Surveycited by 1,015
- 2024Grounding Image Matching in 3D with MASt3Rcited by 983
- 2020Neural Sparse Voxel Fieldscited by 959
38 more
- 2023MVDream: Multi-view Diffusion for 3D Generationcited by 938
- 2020NeRF++: Analyzing and Improving Neural Radiance Fieldscited by 861
- 2021Depth-supervised NeRF: Fewer Views and Faster Training for Freecited by 735
- 2023Scaffold-GS: Structured 3D Gaussians for View-Adaptive Renderingcited by 695
- 2021RegNeRF: Regularizing Neural Radiance Fields for View Synthesis from Sparse Inputscited by 615
- 2022Point-NeRF: Point-based Neural Radiance Fieldscited by 503
- 2022MonoSDF: Exploring Monocular Geometric Cues for Neural Implicit Surface Reconstructioncited by 470
- 2021Putting NeRF on a Diet: Semantically Consistent Few-Shot View Synthesiscited by 464
- 2024ViewCrafter: Taming Video Diffusion Models for High-fidelity Novel View Synthesiscited by 403
- 2023Compact 3D Gaussian Representation for Radiance Fieldcited by 372
- 2021Light Field Networks: Neural Scene Representations with Single-Evaluation Renderingcited by 332
- 2023FreeNeRF: Improving Few-shot Neural Rendering with Free Frequency Regularizationcited by 311
- 2021Dense Depth Priors for Neural Radiance Fields from Sparse Input Viewscited by 302
- 2022Novel View Synthesis with Diffusion Modelscited by 300
- 2023FSGS: Real-Time Few-shot View Synthesis using Gaussian Splattingcited by 279
- 2024DNGaussian: Optimizing Sparse-View 3D Gaussian Radiance Fields with Global-Local Depth Normalizationcited by 256
- 2020Infinite Nature: Perpetual View Generation of Natural Scenes from a Single Imagecited by 243
- 2023LucidDreamer: Domain-free Generation of 3D Gaussian Splatting Scenescited by 231
- 2024Diffusion Model-Based Image Editing: A Surveycited by 212
- 2023COLMAP-Free 3D Gaussian Splattingcited by 193
- 2024VR-GS: A Physical Dynamics-Aware Interactive Gaussian Splatting System in Virtual Realitycited by 179
- 2022SinNeRF: Training Neural Radiance Fields on Complex Scenes from a Single Imagecited by 165
- 2022NeuMesh: Learning Disentangled Neural Mesh-based Implicit Field for Geometry and Texture Editingcited by 142
- 2023Multi-Scale 3D Gaussian Splatting for Anti-Aliased Renderingcited by 138
- 2024CoR-GS: Sparse-View 3D Gaussian Splatting via Co-Regularizationcited by 121
- 2024Pixel-GS: Density Control with Pixel-aware Gradient for 3D Gaussian Splattingcited by 113
- 2023Diffusion Models for Image Restoration and Enhancement: A Comprehensive Surveycited by 109
- 20243DGS-Enhancer: Enhancing Unbounded 3D Gaussian Splatting with View-consistent 2D Diffusion Priorscited by 106
- 2024MVDiffusion++: A Dense High-resolution Multi-view Diffusion Model for Single or Sparse-view 3D Object Reconstructioncited by 95
- 2022InfiniteNature-Zero: Learning Perpetual View Generation of Natural Scenes from Single Imagescited by 82
- 2021FoV-NeRF: Foveated Neural Radiance Fields for Virtual Realitycited by 79
- 2024VR-Splatting: Foveated Radiance Field Rendering via 3D Gaussian Splatting and Neural Pointscited by 29
- 2024A Comprehensive Survey on 3D Content Generationcited by 28
- 2024Advancing Extended Reality with 3D Gaussian Splatting: Innovations and Prospectscited by 17
- 2020Generative View Synthesis: From Single-view Semantics to Novel-view Imagescited by 12
- 2024SfM-Free 3D Gaussian Splatting via Hierarchical Trainingcited by 12
- 2022NeuralPassthrough: Learned Real-Time View Synthesis for VRcited by 8
- 2025See In Detail: Enhancing Sparse-view 3D Gaussian Splatting with Local Depth and Semantic Regularizationcited by 8
50 most cited of 53 All 53 (JSON)
Built on by
Not cited yet.