# HiSplat: Hierarchical 3D Gaussian Splatting for Generalizable Sparse-View Reconstruction

arXiv: https://arxiv.org/abs/2410.06245 · HTML: https://mlsanity.com/paper/2410.06245/

## Key facts

- Cited by 57 papers in the map (data through 14 Oct 2026).
- Builds on 42 of its 52 references that are in the map.
- First posted on arXiv 8 Oct 2024.

## Builds on (most cited first, 30 of 42)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [3D Gaussian Splatting for Real-Time Radiance Field Rendering](https://mlsanity.com/paper/2308.04079/) (2023)
- [Feature Pyramid Networks for Object Detection](https://mlsanity.com/paper/1612.03144/) (2016)
- [FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness](https://mlsanity.com/paper/2205.14135/) (2022)
- [SSD: Single Shot MultiBox Detector](https://mlsanity.com/paper/1512.02325/) (2015)
- [FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioning](https://mlsanity.com/paper/2307.08691/) (2023)
- [Mip-NeRF 360: Unbounded Anti-Aliased Neural Radiance Fields](https://mlsanity.com/paper/2111.12077/) (2021)
- [Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fields](https://mlsanity.com/paper/2103.13415/) (2021)
- [pixelNeRF: Neural Radiance Fields from One or Few Images](https://mlsanity.com/paper/2012.02190/) (2020)
- [UNet++: A Nested U-Net Architecture for Medical Image Segmentation](https://mlsanity.com/paper/1807.10165/) (2018)
- [D-NeRF: Neural Radiance Fields for Dynamic Scenes](https://mlsanity.com/paper/2011.13961/) (2020)
- [Spatial Transformer Networks](https://mlsanity.com/paper/1506.02025/) (2015)
- [The Replica Dataset: A Digital Replica of Indoor Spaces](https://mlsanity.com/paper/1906.05797/) (2019)
- [MVSNet: Depth Inference for Unstructured Multi-view Stereo](https://mlsanity.com/paper/1804.02505/) (2018)
- [Deformable 3D Gaussians for High-Fidelity Monocular Dynamic Scene Reconstruction](https://mlsanity.com/paper/2309.13101/) (2023)
- [Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition](https://mlsanity.com/paper/1406.4729/) (2014)
- [pixelSplat: 3D Gaussian Splats from Image Pairs for Scalable Generalizable 3D Reconstruction](https://mlsanity.com/paper/2312.12337/) (2023)
- [KiloNeRF: Speeding up Neural Radiance Fields with Thousands of Tiny MLPs](https://mlsanity.com/paper/2103.13744/) (2021)
- [Mip-Splatting: Alias-free 3D Gaussian Splatting](https://mlsanity.com/paper/2311.16493/) (2023)
- [MVSplat: Efficient 3D Gaussian Splatting from Sparse Multi-View Images](https://mlsanity.com/paper/2403.14627/) (2024)
- [In-Place Scene Labelling and Understanding with Implicit Scene Representation](https://mlsanity.com/paper/2103.15875/) (2021)
- [Splatter Image: Ultra-Fast Single-View 3D Reconstruction](https://mlsanity.com/paper/2312.13150/) (2023)
- [GMFlow: Learning Optical Flow via Global Matching](https://mlsanity.com/paper/2111.13680/) (2021)
- [NAS-FPN: Learning Scalable Feature Pyramid Architecture for Object Detection](https://mlsanity.com/paper/1904.07392/) (2019)
- [PGSR: Planar-based Gaussian Splatting for Efficient and High-Fidelity Surface Reconstruction](https://mlsanity.com/paper/2406.06521/) (2024)
- [Unifying Flow, Stereo and Depth Estimation](https://mlsanity.com/paper/2211.05783/) (2022)

## Built on by (most cited first, 30 of 57)

- [DiffPano: Scalable and Consistent Text to Panorama Generation with Spherical Epipolar-Aware Diffusion](https://mlsanity.com/paper/2410.24203/) (2024)
- [PhysFlow: Unleashing the Potential of Multi-modal Foundation Models and Video Diffusion for 4D Dynamic Physical Scene Simulation](https://mlsanity.com/paper/2411.14423/) (2024)
- [Advances in Feed-Forward 3D Reconstruction and View Synthesis: A Survey](https://mlsanity.com/paper/2507.14501/) (2025)
- [CityGS-X: A Scalable Architecture for Efficient and Geometrically Accurate Large-Scale Scene Reconstruction](https://mlsanity.com/paper/2503.23044/) (2025)
- [Splatter-360: Generalizable 360circ Gaussian Splatting for Wide-baseline Panoramic Images](https://mlsanity.com/paper/2412.06250/) (2024)
- [PanSplat: 4K Panorama Synthesis with Feed-Forward Gaussian Splatting](https://mlsanity.com/paper/2412.12096/) (2024)
- [MonoSplat: Generalizable 3D Gaussian Splatting from Monocular Depth Foundation Models](https://mlsanity.com/paper/2505.15185/) (2025)
- [FreeSplat++: Generalizable 3D Gaussian Splatting for Efficient Indoor Scene Reconstruction](https://mlsanity.com/paper/2503.22986/) (2025)
- [SkySplat: Generalizable 3D Gaussian Splatting from Multi-Temporal Sparse Satellite Images](https://mlsanity.com/paper/2508.09479/) (2025)
- [TokenSplat: Token-aligned 3D Gaussian Splatting for Feed-forward Pose-free Reconstruction](https://mlsanity.com/paper/2603.00697/) (2026)
- [LongSplat: Online Generalizable 3D Gaussian Splatting from Long Sequence Images](https://mlsanity.com/paper/2507.16144/) (2025)
- [One2Scene: Geometric Consistent Explorable 3D Scene Generation from a Single Image](https://mlsanity.com/paper/2602.19766/) (2026)
- [PanoSplatt3R: Leveraging Perspective Pretraining for Generalized Unposed Wide-Baseline Panorama Reconstruction](https://mlsanity.com/paper/2507.21960/) (2025)
- [Anti-Aliased 2D Gaussian Splatting](https://mlsanity.com/paper/2506.11252/) (2025)
- [TextSplat: Text-Guided Semantic Fusion for Generalizable Gaussian Splatting](https://mlsanity.com/paper/2504.09588/) (2025)
- [X-GRM: Large Gaussian Reconstruction Model for Sparse-view X-rays to Computed Tomography](https://mlsanity.com/paper/2505.15235/) (2025)
- [Feed-Forward 3D Scene Modeling: A Problem-Driven Perspective](https://mlsanity.com/paper/2604.14025/) (2026)
- [Learning Efficient Fuse-and-Refine for Feed-Forward 3D Gaussian Splatting](https://mlsanity.com/paper/2503.14698/) (2025)
- [IDESplat: Iterative Depth Probability Estimation for Generalizable 3D Gaussian Splatting](https://mlsanity.com/paper/2601.03824/) (2026)
- [RecGen3D: Reconstruction-Guided 3D Generation in a Shared Canonical Space](https://mlsanity.com/paper/2604.01479/) (2026)
- [Sparse-View 3D Reconstruction: Recent Advances and Open Challenges](https://mlsanity.com/paper/2507.16406/) (2025)
- [H3R: Hybrid Multi-view Correspondence for Generalizable 3D Reconstruction](https://mlsanity.com/paper/2508.03118/) (2025)
- [GaMO: Geometry-aware Multi-view Diffusion Outpainting for Sparse-View 3D Reconstruction](https://mlsanity.com/paper/2512.25073/) (2025)
- [Uncertainty-Aware Diffusion Guided Refinement of 3D Scenes](https://mlsanity.com/paper/2503.15742/) (2025)
- [JointSplat: Probabilistic Joint Flow-Depth Optimization for Sparse-View Gaussian Splatting](https://mlsanity.com/paper/2506.03872/) (2025)
- [TextureSplat: Per-Primitive Texture Mapping for Reflective Gaussian Splatting](https://mlsanity.com/paper/2506.13348/) (2025)
- [GaussianLens: Localized High-Resolution Reconstruction via On-Demand Gaussian Densification](https://mlsanity.com/paper/2509.25603/) (2025)
- [SRSplat: Feed-Forward Super-Resolution Gaussian Splatting from Sparse Multi-View Images](https://mlsanity.com/paper/2511.12040/) (2025)
- [On-the-fly Large-scale 3D Reconstruction from Multi-Camera Rigs](https://mlsanity.com/paper/2512.08498/) (2025)
- [G3Splat: Geometrically Consistent Generalizable Gaussian Splatting](https://mlsanity.com/paper/2512.17547/) (2025)

## Abstract (arXiv)

Reconstructing 3D scenes from multiple viewpoints is a fundamental task in stereo vision. Recently, advances in generalizable 3D Gaussian Splatting have enabled high-quality novel view synthesis for unseen scenes from sparse input views by feed-forward predicting per-pixel Gaussian parameters without extra optimization. However, existing methods typically generate single-scale 3D Gaussians, which lack representation of both large-scale structure and texture details, resulting in mislocation and artefacts. In this paper, we propose a novel framework, HiSplat, which introduces a hierarchical manner in generalizable 3D Gaussian Splatting to construct hierarchical 3D Gaussians via a coarse-to-fine strategy. Specifically, HiSplat generates large coarse-grained Gaussians to capture large-scale structures, followed by fine-grained Gaussians to enhance delicate texture details. To promote inter-scale interactions, we propose an Error Aware Module for Gaussian compensation and a Modulating Fusion Module for Gaussian repair. Our method achieves joint optimization of hierarchical representations, allowing for novel view synthesis using only two-view reference images. Comprehensive experiments on various datasets demonstrate that HiSplat significantly enhances reconstruction quality and cross-dataset generalization compared to prior single-scale methods. The corresponding ablation study and analysis of different-scale 3D Gaussians reveal the mechanism behind the effectiveness. Project website: https://open3dvlab.github.io/HiSplat/