360^circ Reconstruction From a Single Image Using Space Carved Outpainting
Not on the map. Why?
Abstract (arXiv)
We introduce POP3D, a novel framework that creates a full 360^circ-view 3D model from a single image. POP3D resolves two prominent issues that limit the single-view reconstruction. Firstly, POP3D offers substantial generalizability to arbitrary categories, a trait that previous methods struggle to achieve. Secondly, POP3D further improves reconstruction fidelity and naturalness, a crucial aspect that concurrent works fall short of. Our approach marries the strengths of four primary components: (1) a monocular depth and normal predictor that serves to predict crucial geometric cues, (2) a space carving method capable of demarcating the potentially unseen portions of the target object, (3) a generative model pre-trained on a large-scale image dataset that can complete unseen regions of the target, and (4) a neural implicit surface reconstruction method tailored in reconstructing objects using RGB images along with monocular geometric cues. The combination of these components enables POP3D to readily generalize across various in-the-wild images and generate state-of-the-art reconstructions, outperforming similar works by a significant margin. Project page: urlhttp://cg.postech.ac.kr/research/POP3D
Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 60 of its 72 references that are in the map. First posted on arXiv 19 Sep 2023.
Builds on
60 of 72 references.
References · 60
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
- 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
- 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
- 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,177
- 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,152
- 2022DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generationcited by 3,300
- 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,025
- 2022An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversioncited by 2,245
- 2021Mip-NeRF 360: Unbounded Anti-Aliased Neural Radiance Fieldscited by 2,032
- 2021Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fieldscited by 1,964
- 2020pixelNeRF: Neural Radiance Fields from One or Few Imagescited by 1,584
- 2022Objaverse: A Universe of Annotated 3D Objectscited by 1,540
38 more
- 2023Zero-1-to-3: Zero-shot One Image to 3D Objectcited by 1,489
- 2021Volume Rendering of Neural Implicit Surfacescited by 996
- 2019Scene Representation Networks: Continuous 3D-Structure-Aware Neural Scene Representationscited by 864
- 2020NeRF++: Analyzing and Improving Neural Radiance Fieldscited by 861
- 2019PIFu: Pixel-Aligned Implicit Function for High-Resolution Clothed Human Digitizationcited by 818
- 2019Differentiable Volumetric Rendering: Learning Implicit 3D Representations without 3D Supervisioncited by 700
- 2022Point-E: A System for Generating 3D Point Clouds from Complex Promptscited by 699
- 2023ZoeDepth: Zero-shot Transfer by Combining Relative and Metric Depthcited by 657
- 2018Pixel2Mesh: Generating 3D Mesh Models from Single RGB Imagescited by 651
- 20163D-R2N2: A Unified Approach for Single and Multi-view 3D Object Reconstructioncited by 588
- 2021Ref-NeRF: Structured View-Dependent Appearance for Neural Radiance Fieldscited by 575
- 2018AtlasNet: A Papier-Mach'e Approach to Learning 3D Surface Generationcited by 539
- 2023Exploiting Diffusion Prior for Real-World Image Super-Resolutioncited by 489
- 2022MonoSDF: Exploring Monocular Geometric Cues for Neural Implicit Surface Reconstructioncited by 470
- 2021Putting NeRF on a Diet: Semantically Consistent Few-Shot View Synthesiscited by 464
- 2019SAL: Sign Agnostic Learning of Shapes from Raw Datacited by 390
- 2023Make-It-3D: High-Fidelity 3D Creation from A Single Image with Diffusion Priorcited by 343
- 2021Omnidata: A Scalable Pipeline for Making Multi-Task Mid-Level Vision Datasets from 3D Scanscited by 336
- 2018Learning Category-Specific Mesh Reconstruction from Image Collectionscited by 317
- 2022NeuS2: Fast Learning of Neural Implicit Surfaces for Multi-view Reconstructioncited by 314
- 2022Rodin: A Generative Model for Sculpting 3D Digital Avatars Using Diffusioncited by 309
- 2023RealFusion: 360deg Reconstruction of Any Object from a Single Imagecited by 306
- 2021Dense Depth Priors for Neural Radiance Fields from Sparse Input Viewscited by 302
- 2022Novel View Synthesis with Diffusion Modelscited by 300
- 20223D Neural Field Generation using Triplane Diffusioncited by 274
- 2023Generative Novel View Synthesis with 3D-Aware Diffusion Modelscited by 274
- 2023Text2Room: Extracting Textured 3D Meshes from 2D Text-to-Image Modelscited by 245
- 2023DreamBooth3D: Subject-Driven Text-to-3D Generationcited by 235
- 2021Scene Representation Transformer: Geometry-Free Novel View Synthesis Through Set-Latent Scene Representationscited by 227
- 2022SparseFusion: Distilling View-conditioned Diffusion for 3D Reconstructioncited by 216
- 20203D Photography using Context-aware Layered Depth Inpaintingcited by 206
- 2021CodeNeRF: Disentangled Neural Radiance Fields for Object Categoriescited by 183
- 2023NerfDiff: Single-image View Synthesis with NeRF-guided Distillation from 3D-aware Diffusioncited by 180
- 2016Learning a Predictable and Generative Vector Representation for Objectscited by 173
- 2018Escaping Plato's Cave: 3D Shape From Adversarial Renderingcited by 170
- 2019Pix2Vox: Context-aware 3D Reconstruction from Single and Multi-view Imagescited by 164
- 2022NeRDi: Single-View NeRF Synthesis with Language-Guided Diffusion as General Image Priorscited by 156
- 2022NeuralLift-360: Lifting An In-the-wild 2D Photo to A 3D Object with 360deg Viewscited by 148
50 most cited of 60 All 60 (JSON)
Built on by
Citing papers per year: 2024: 1; 2025: 1.
2 papers.