VolFill: Single-View Amodal 3D Scene Reconstruction with Volumetric Flow Matching
On the map: Diffusion models → 3D, point cloud and scene generation
Abstract (arXiv)
Reconstructing the complete geometry of a scene from a single RGB image remains challenging - especially when inferring hidden structures where visual evidence is incomplete. We introduce VolFill, a generative framework that predicts the 3D structure of the complete scene rather than relying on traditional pixel-aligned regression. Our method utilizes a hybrid 3D VAE to compress sparse truncated unsigned distance function grids into a compact latent space, paired with a latent Diffusion Transformer that denoises this representation to recover the complete scene. We condition the generation on geometry foundation models, leveraging rich spatial priors for robust reasoning. Unlike existing methods limited by per-ray constraints or unstructured point-cloud queries, VolFill provides a structured representation that supports direct surface extraction and occupancy queries at scale. Extensive experiments on the SCRREAM and NRGB-D datasets demonstrate that our approach significantly outperforms current baselines, providing a robust foundation for holistic spatial understanding.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in 3D, point cloud and scene generation, one of 14 directions in Diffusion models. Builds on 89 of its 99 references that are in the map. First posted on arXiv 29 May 2026.
Builds on
89 of 99 references.
Closest ancestors · 22
Same direction, newest first
- 2026Repurposing Geometric Foundation Models for Multi-view Diffusion23 Mar
- 2025Pixel-Perfect Depth with Semantics-Prompted Diffusion Transformers8 Oct
- 2025Reconstruct, Inpaint, Test-Time Finetune: Dynamic Novel-view Synthesis from Monocular Videos16 Jul
- 2025Geometry Forcing: Marrying Video Diffusion and 3D Representation for Consistent World Modeling10 Jul
- 2025Stable Virtual Camera: Generative View Synthesis with Diffusion Models18 Mar
- 2025TrajectoryCrafter: Redirecting Camera Trajectory for Monocular Videos via Diffusion Models7 Mar
- 2025TripoSG: High-Fidelity 3D Shape Synthesis using Large-Scale Rectified Flow Models10 Feb
- 20244Real-Video: Learning Generalizable Photo-Realistic 4D Video Diffusion5 Dec
- 2024SharpDepth: Sharpening Metric Depth Predictions Using Diffusion Distillation27 Nov
- 2024Fine-Tuning Image-Conditional Diffusion Models is Easier than You Think17 Sep
- 2024ViewCrafter: Taming Video Diffusion Models for High-fidelity Novel View Synthesis3 Sep
- 2024CAT3D: Create Anything in 3D with Multi-View Diffusion Models16 May
10 more
- 2024GeoWizard: Unleashing the Diffusion Priors for 3D Geometry Estimation from a Single Image18 Mar
- 2023Repurposing Diffusion-Based Image Generators for Monocular Depth Estimation4 Dec
- 2023Instant3D: Fast Text-to-3D with Sparse-View Generation and Large Reconstruction Model10 Nov
- 2023Wonder3D: Single Image to 3D using Cross-Domain Diffusion23 Oct
- 2023SyncDreamer: Generating Multiview-consistent Images from a Single-view Image7 Sep
- 2023MVDream: Multi-view Diffusion for 3D Generation31 Aug
- 2023Single-Stage Diffusion NeRF: A Unified Approach to 3D Generation and Reconstruction13 Apr
- 2023Zero-1-to-3: Zero-shot One Image to 3D Object20 Mar
- 2022Point-E: A System for Generating 3D Point Clouds from Complex Prompts16 Dec
- 2022DiffRF: Rendering-Guided 3D Radiance Field Diffusion2 Dec
Other references · 67
- 2017Attention Is All You Needcited by 60,361
- 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
- 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,226
- 2022Scalable Diffusion Models with Transformerscited by 6,054
- 2022Flow Matching for Generative Modelingcited by 5,702
- 2023SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesiscited by 4,356
- 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,700
- 2016Gaussian Error Linear Units (GELUs)cited by 3,502
- 2020Implicit Neural Representations with Periodic Activation Functionscited by 2,238
- 2023DUSt3R: Geometric 3D Vision Made Easycited by 1,567
- 2025VGGT: Visual Geometry Grounded Transformercited by 1,539
- 2019Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfercited by 1,495
38 more
- 2019Root Mean Square Layer Normalizationcited by 1,480
- 2024Depth Anything V2cited by 1,422
- 2024Depth Anything: Unleashing the Power of Large-Scale Unlabeled Datacited by 1,360
- 2014Depth Map Prediction from a Single Image using a Multi-Scale Deep Networkcited by 1,274
- 20194D Spatio-Temporal ConvNets: Minkowski Convolutional Neural Networkscited by 1,208
- 2022Building Normalizing Flows with Stochastic Interpolantscited by 995
- 2023LRM: Large Reconstruction Model for Single Image to 3Dcited by 849
- 2024Structured 3D Latents for Scalable and Versatile 3D Generationcited by 770
- 2023ScanNet++: A High-Fidelity Dataset of 3D Indoor Scenescited by 764
- 2024LGM: Large Multi-View Gaussian Model for High-Resolution 3D Content Creationcited by 739
- 2018Deep Ordinal Regression Network for Monocular Depth Estimationcited by 723
- 2019Differentiable Volumetric Rendering: Learning Implicit 3D Representations without 3D Supervisioncited by 700
- 2025Depth Anything 3: Recovering the Visual Space from Any Viewscited by 692
- 2023ZoeDepth: Zero-shot Transfer by Combining Relative and Metric Depthcited by 657
- 2020AdaBins: Depth Estimation using Adaptive Binscited by 601
- 2016Semantic Scene Completion from a Single Depth Imagecited by 472
- 2024CameraCtrl: Enabling Camera Control for Text-to-Video Generationcited by 460
- 2024Depth Pro: Sharp Monocular Metric Depth in Less Than a Secondcited by 404
- 2024Metric3Dv2: A Versatile Monocular Geometric Foundation Model for Zero-shot Metric Depth and Surface Normal Estimationcited by 395
- 2019From Big to Small: Multi-Scale Local Planar Guidance for Monocular Depth Estimationcited by 382
- 2024UniDepth: Universal Monocular Metric Depth Estimationcited by 382
- 20203D-FRONT: 3D Furnished Rooms with layOuts and semaNTicscited by 380
- 2023Metric3D: Towards Zero-shot Metric 3D Prediction from A Single Imagecited by 365
- 2021MonoScene: Monocular 3D Semantic Scene Completioncited by 363
- 2024GS-LRM: Large Reconstruction Model for 3D Gaussian Splattingcited by 349
- 2025GEN3C: 3D-Informed World-Consistent Video Generation with Precise Camera Controlcited by 341
- 2023SurroundOcc: Multi-Camera 3D Occupancy Prediction for Autonomous Drivingcited by 316
- 2025MapAnything: Universal Feed-Forward Metric 3D Reconstructioncited by 314
- 2021Neural RGB-D Surface Reconstructioncited by 313
- 2023VoxFormer: Sparse Voxel Transformer for Camera-based 3D Semantic Scene Completioncited by 313
- 2024GRM: Large Gaussian Reconstruction Model for Efficient 3D Reconstruction and Generationcited by 301
- 2024MoGe: Unlocking Accurate Monocular Geometry Estimation for Open-Domain Images with Optimal Training Supervisioncited by 299
- 2022SDFusion: Multimodal 3D Shape Completion, Reconstruction, and Generationcited by 288
- 2020Query-Key Normalization for Transformerscited by 280
- 2025MoGe-2: Accurate Monocular Geometry with Metric Scale and Sharp Detailscited by 261
- 2023Uni3D: Exploring Unified 3D Representation at Scalecited by 237
- 2017Submanifold Sparse Convolutional Networkscited by 223
- 2023XCube: Large-Scale 3D Generative Modeling using Sparse Voxel Hierarchiescited by 189
50 most cited of 67 All 67 (JSON)
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2024Coherent 3D Scene Diffusion From a Single RGB Image
- 2026Feed-Forward 3D Scene Modeling: A Problem-Driven Perspective
- 2026ReconSplat: Generalizable 3D Scene Reconstruction Beyond Observed Views
- 2025G4Splat: Geometry-Guided Gaussian Splatting with Generative Prior
- 2026Seen2Scene: Completing Realistic 3D Scenes with Visibility-Guided Flow
- 2026Gen3R: 3D Scene Generation Meets Feed-Forward Reconstruction
- 2024VI3DRM:Towards meticulous 3D Reconstruction from Sparse Views via Photo-Realistic Novel View Synthesis
- 2026FILIGREE3D: Scaling Sparse Latent Flow Matching for Ultra-High-Resolution Image-to-3D Generation