# Dust to Tower: Coarse-to-Fine Photo-Realistic Scene Reconstruction from Sparse Uncalibrated Images

arXiv: https://arxiv.org/abs/2412.19518 · HTML: https://mlsanity.com/paper/2412.19518/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 91 references that are in the map.
- First posted on arXiv 27 Dec 2024.

## Builds on (most cited first, 22 of 22)

- [Depth Anything V2](https://mlsanity.com/paper/2406.09414/) (2024)
- [NeRF++: Analyzing and Improving Neural Radiance Fields](https://mlsanity.com/paper/2010.07492/) (2020)
- [ModelScope Text-to-Video Technical Report](https://mlsanity.com/paper/2308.06571/) (2023)
- [MVSplat: Efficient 3D Gaussian Splatting from Sparse Multi-View Images](https://mlsanity.com/paper/2403.14627/) (2024)
- [ViewCrafter: Taming Video Diffusion Models for High-fidelity Novel View Synthesis](https://mlsanity.com/paper/2409.02048/) (2024)
- [gsplat: An Open-Source Library for Gaussian Splatting](https://mlsanity.com/paper/2409.06765/) (2024)
- [No Pose, No Problem: Surprisingly Simple 3D Gaussian Splats from Sparse Unposed Images](https://mlsanity.com/paper/2410.24207/) (2024)
- [CRM: Single Image to 3D Textured Mesh with Convolutional Reconstruction Model](https://mlsanity.com/paper/2403.05034/) (2024)
- [NeRF--: Neural Radiance Fields Without Known Camera Parameters](https://mlsanity.com/paper/2102.07064/) (2021)
- [ReconX: Reconstruct Any Scene from Sparse Views with Video Diffusion Model](https://mlsanity.com/paper/2408.16767/) (2024)
- [SparseGS: Sparse View Synthesis using 3D Gaussian Splatting](https://mlsanity.com/paper/2312.00206/) (2023)
- [3DGS-Enhancer: Enhancing Unbounded 3D Gaussian Splatting with View-consistent 2D Diffusion Priors](https://mlsanity.com/paper/2410.16266/) (2024)
- [GARF: Gaussian Activated Radiance Fields for High Fidelity Reconstruction and Pose Estimation](https://mlsanity.com/paper/2204.05735/) (2022)
- [InstantSplat: Sparse-view Gaussian Splatting in Seconds](https://mlsanity.com/paper/2403.20309/) (2024)
- [AutoSplat: Constrained Gaussian Splatting for Autonomous Driving Scene Reconstruction](https://mlsanity.com/paper/2407.02598/) (2024)
- [SplatLoc: 3D Gaussian Splatting-based Visual Localization for Augmented Reality](https://mlsanity.com/paper/2409.14067/) (2024)
- [LM-Gaussian: Boost Sparse-view 3D Gaussian Splatting with Large Model Priors](https://mlsanity.com/paper/2409.03456/) (2024)
- [Real3D: Scaling Up Large Reconstruction Models with Real-World Images](https://mlsanity.com/paper/2406.08479/) (2024)
- [FewViewGS: Gaussian Splatting with Few View Matching and Multi-stage Training](https://mlsanity.com/paper/2411.02229/) (2024)
- [Correspondence-Guided SfM-Free 3D Gaussian Splatting for NVS](https://mlsanity.com/paper/2408.08723/) (2024)
- [Multi-View Large Reconstruction Model via Geometry-Aware Positional Encoding and Attention](https://mlsanity.com/paper/2406.07648/) (2024)
- [PRISM: PRogressive dependency maxImization for Scale-invariant image Matching](https://mlsanity.com/paper/2408.03598/) (2024)

## Built on by (most cited first, 8 of 8)

- [Taming Video Models for 3D and 4D Generation via Zero-Shot Camera Control](https://mlsanity.com/paper/2509.15130/) (2025)
- [G4Splat: Geometry-Guided Gaussian Splatting with Generative Prior](https://mlsanity.com/paper/2510.12099/) (2025)
- [Digital Twin Generation from Visual Data: A Survey](https://mlsanity.com/paper/2504.13159/) (2025)
- [Sparse-View 3D Reconstruction: Recent Advances and Open Challenges](https://mlsanity.com/paper/2507.16406/) (2025)
- [MapDream: Task-Driven Map Learning for Vision-Language Navigation](https://mlsanity.com/paper/2602.00222/) (2026)
- [Multi-view Normal and Distance Guidance Gaussian Splatting for Surface Reconstruction](https://mlsanity.com/paper/2508.07701/) (2025)
- [Pose-Free Omnidirectional Gaussian Splatting for 360-Degree Videos with Consistent Depth Priors](https://mlsanity.com/paper/2603.23324/) (2026)
- [PR-IQA: Partial-Reference Image Quality Assessment for Diffusion-Based Novel View Synthesis](https://mlsanity.com/paper/2604.04576/) (2026)

## Abstract (arXiv)

Photo-realistic scene reconstruction from sparse-view, uncalibrated images is highly required in practice. Although some successes have been made, existing methods are either Sparse-View but require accurate camera parameters (i.e., intrinsic and extrinsic), or SfM-free but need densely captured images. To combine the advantages of both methods while addressing their respective weaknesses, we propose Dust to Tower (D2T), an accurate and efficient coarse-to-fine framework to optimize 3DGS and image poses simultaneously from sparse and uncalibrated images. Our key idea is to first construct a coarse model efficiently and subsequently refine it using warped and inpainted images at novel viewpoints. To do this, we first introduce a Coarse Construction Module (CCM) which exploits a fast Multi-View Stereo model to initialize a 3D Gaussian Splatting (3DGS) and recover initial camera poses. To refine the 3D model at novel viewpoints, we propose a Confidence Aware Depth Alignment (CADA) module to refine the coarse depth maps by aligning their confident parts with estimated depths by a Mono-depth model. Then, a Warped Image-Guided Inpainting (WIGI) module is proposed to warp the training images to novel viewpoints by the refined depth maps, and inpainting is applied to fulfill the ``holes" in the warped images caused by view-direction changes, providing high-quality supervision to further optimize the 3D model and the camera poses. Extensive experiments and ablation studies demonstrate the validity of D2T and its design choices, achieving state-of-the-art performance in both tasks of novel view synthesis and pose estimation while keeping high efficiency. Codes will be publicly available.