mlsanityData to 7 Oct 2026

Zero-Shot Scene Reconstruction from Single Images with Deep Prior Assembly

arXiv 2410.15971 ↗First posted 21 Oct 2024cs.CVcited by 34

Not on the map. Why?

Abstract (arXiv)

Large language and vision models have been leading a revolution in visual computing. By greatly scaling up sizes of data and model parameters, the large models learn deep priors which lead to remarkable performance in various tasks. In this work, we present deep prior assembly, a novel framework that assembles diverse deep priors from large models for scene reconstruction from single images in a zero-shot manner. We show that this challenging task can be done without extra knowledge but just simply generalizing one deep prior in one sub-task. To this end, we introduce novel methods related to poses, scales, and occlusion parsing which are keys to enable deep priors to work together in a robust way. Deep prior assembly does not require any 3D or 2D data-driven training in the task and demonstrates superior performance in generalizing priors to open-world scenes. We conduct evaluations on various datasets, and report analysis, numerical and visual comparisons with the latest methods to show our superiority. Project page: https://junshengzhou.github.io/DeepPriorAssembly.

Key facts. Cited by 34 papers in the map (data through 7 Oct 2026). Builds on 62 of its 79 references that are in the map. First posted on arXiv 21 Oct 2024.

Builds on

62 of 79 references.

References · 62

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020Language Models are Few-Shot Learnerscited by 32,199
  3. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  4. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  5. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
  6. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  7. 2019RoBERTa: A Robustly Optimized BERT Pretraining Approachcited by 13,266
  8. 2023Segment Anythingcited by 8,158
  9. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
  10. 2017Mask R-CNNcited by 7,259
  11. 2023Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detectioncited by 3,274
  12. 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,088
38 more
  1. 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,880
  2. 2019DeepSDF: Learning Continuous Signed Distance Functions for Shape Representationcited by 2,682
  3. 2021BEiT: BERT Pre-Training of Image Transformerscited by 2,360
  4. 2018Occupancy Networks: Learning 3D Reconstruction in Function Spacecited by 2,037
  5. 2021SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equationscited by 1,903
  6. 2021NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstructioncited by 1,701
  7. 2019Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfercited by 1,484
  8. 2024Depth Anything V2cited by 1,385
  9. 2024Depth Anything: Unleashing the Power of Large-Scale Unlabeled Datacited by 1,346
  10. 2014Depth Map Prediction from a Single Image using a Multi-Scale Deep Networkcited by 1,264
  11. 2021Volume Rendering of Neural Implicit Surfacescited by 990
  12. 2019The Replica Dataset: A Digital Replica of Indoor Spacescited by 885
  13. 2023EVA-CLIP: Improved Training Techniques for CLIP at Scalecited by 786
  14. 2022Point-E: A System for Generating 3D Point Clouds from Complex Promptscited by 694
  15. 2022EVA: Exploring the Limits of Masked Visual Representation Learning at Scalecited by 685
  16. 2021UNISURF: Unifying Neural Implicit Surfaces and Radiance Fields for Multi-View Reconstructioncited by 643
  17. 2023One-2-3-45: Any Single Image to 3D Mesh in 45 Seconds without Per-Shape Optimizationcited by 555
  18. 2022MonoSDF: Exploring Monocular Geometric Cues for Neural Implicit Surface Reconstructioncited by 467
  19. 20203D-FRONT: 3D Furnished Rooms with layOuts and semaNTicscited by 375
  20. 2023Metric3D: Towards Zero-shot Metric 3D Prediction from A Single Imagecited by 357
  21. 2021Omnidata: A Scalable Pipeline for Making Multi-Task Mid-Level Vision Datasets from 3D Scanscited by 336
  22. 2023PointLLM: Empowering Large Language Models to Understand Point Cloudscited by 326
  23. 2023Repurposing Diffusion-Based Image Generators for Monocular Depth Estimationcited by 323
  24. 2021Neural RGB-D Surface Reconstructioncited by 312
  25. 20203D-FUTURE: 3D Furniture shape with TextUREcited by 306
  26. 2023Uni3D: Exploring Unified 3D Representation at Scalecited by 231
  27. 2018Pix3D: Dataset and Methods for Single-Image 3D Shape Modelingcited by 218
  28. 2021Shape As Points: A Differentiable Poisson Solvercited by 186
  29. 2020Total3DUnderstanding: Joint Layout, Object Pose and Mesh Reconstruction for Indoor Scenes from a Single Imagecited by 170
  30. 2022NeuRIS: Neural Reconstruction of Indoor Scenes Using Normal Priorscited by 125
  31. 20223D Common Corruptions and Data Augmentationcited by 120
  32. 2021Panoptic 3D Scene Reconstruction From a Single RGB Imagecited by 70
  33. 2024Neural Signed Distance Function Inference through Splatting 3D Gaussians Pulled on Zero-Level Setcited by 47
  34. 2022CAP-UDF: Learning Unsigned Distance Functions Progressively from Raw Point Clouds with Consistency-Aware Field Optimizationcited by 46
  35. 20223D Shape Reconstruction from 2D Images with Disentangled Attribute Flowcited by 45
  36. 2023Towards Better Gradient Consistency for Neural Signed Distance Functions via Level Set Alignmentcited by 39
  37. 2022Fast Learning Radiance Fields by Shooting Much Fewer Rayscited by 38
  38. 2022NeAF: Learning Neural Angle Fields for Point Normal Estimationcited by 38

50 most cited of 62 All 62 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 4; 2025: 18; 2026: 12.

34 papers.

Sort:

By area:

  1. 2024MIDI: Multi-Instance Diffusion for Single Image to 3D Scene Generationcited by 113
  2. 2025PartCrafter: Structured 3D Mesh Generation via Compositional Latent Diffusion Transformerscited by 78
  3. 2025Scenethesis: A Language and Vision Agentic Framework for 3D Scene Generationcited by 65
  4. 2024DiffGS: Functional Gaussian Splatting Diffusioncited by 50
  5. 2024Binocular-Guided 3D Gaussian Splatting with View Consistency for Sparse View Synthesiscited by 41
  6. 2025AutoPartGen: Autogressive 3D Part Generation and Discoverycited by 33
  7. 2025DepR: Depth Guided Single-view Scene Reconstruction with Instance-level Diffusioncited by 23
  8. 2025MonoInstance: Enhancing Monocular Priors via Multi-view Instance Alignment for Neural Rendering and Reconstructioncited by 15
  9. 2024Diorama: Unleashing Zero-shot Single-view 3D Indoor Scene Modelingcited by 10
  10. 2025UVGS: Reimagining Unstructured 3D Gaussian Splatting using UV Mappingcited by 8
  11. 2025Learning Bijective Surface Parameterization for Inferring Signed Distance Functions from Sparse Point Clouds with Grid Deformationcited by 7
  12. 2025Digital Twin Generation from Visual Data: A Surveycited by 7
22 more
  1. 2025GAP: Gaussianize Any Point Clouds with Text Guidancecited by 7
  2. 2025Towards Geometric and Textural Consistency 3D Scene Generation via Single Image-guided Model Generation and Layout Optimizationcited by 6
  3. 2025I-Scene: 3D Instance Models are Implicit Generalizable Spatial Learnerscited by 6
  4. 2025MoCA: Mixture-of-Components Attention for Scalable Compositional 3D Generationcited by 5
  5. 2025Recent Advances in 3D Object and Scene Generation: A Surveycited by 4
  6. 2025ZeroScene: A Zero-Shot Framework for 3D Scene Generation from a Single Image and Controllable Texture Editingcited by 4
  7. 2025U-CAN: Unsupervised Point Cloud Denoising with Consistency-Aware Noise2Noise Matchingcited by 3
  8. 2026PixARMesh: Autoregressive Mesh-Native Single-View Scene Reconstructioncited by 3
  9. 2026AdaSFormer: Adaptive Serialized Transformers for Monocular Semantic Scene Completion from Indoor Environmentscited by 3
  10. 2026φ-Scene: Physically Grounded Image-to-3D Scene Reconstructioncited by 3
  11. 2025DreamAnywhere: Object-Centric Panoramic 3D Scene Generationcited by 2
  12. 2025UnPose: Uncertainty-Guided Diffusion Priors for Zero-Shot Pose Estimationcited by 2
  13. 2026Lucida: Parse, Generate, and Place for Composable Real-to-Sim Scene Modelingcited by 2
  14. 2026Seeing Through Clutter: Structured 3D Scene Reconstruction via Iterative Object Removalcited by 1
  15. 2026DM-CFO: A Diffusion Model for Compositional 3D Tooth Generation with Collision-Free Optimizationcited by 1
  16. 2026SGAD-SLAM: Splatting Gaussians at Adjusted Depth for Better Radiance Fields in RGBD SLAMcited by 1
  17. 2026JRM: Joint Reconstruction Model for Multiple Objects without Alignmentcited by 1
  18. 2026Pose-Aware Diffusion for 3D Generationcited by 1
  19. 2025CC-FMO: Camera-Conditioned Zero-Shot Single Image to 3D Scene Generation with Foundation Model Orchestrationcited by 0
  20. 2026TIMI: Training-Free Image-to-3D Multi-Instance Generation with Spatial Fidelitycited by 0
  21. 2026HIVE-3D: Hierarchical Voxel Enhancement for High-Quality 3D Scene Generationcited by 0
  22. 2026Mira-Scene: Pixel-Aligned Layouts for Generative 3D Scene Reconstructioncited by 0