mlsanityData to 7 Oct 2026

Learning to Predict Scene-Level Implicit 3D from Posed RGBD Data

arXiv 2306.08671 ↗First posted 14 Jun 2023cs.CVcited by 2

Not on the map. Why?

Abstract (arXiv)

We introduce a method that can learn to predict scene-level implicit functions for 3D reconstruction from posed RGBD data. At test time, our system maps a previously unseen RGB image to a 3D reconstruction of a scene via implicit functions. While implicit functions for 3D reconstruction have often been tied to meshes, we show that we can train one using only a set of posed RGBD images. This setting may help 3D reconstruction unlock the sea of accelerometer+RGBD data that is coming with new phones. Our system, D2-DRDF, can match and sometimes outperform current methods that use mesh supervision and shows better robustness to sparse data.

Key facts. Cited by 2 papers in the map (data through 7 Oct 2026). Builds on 49 of its 66 references that are in the map. First posted on arXiv 14 Jun 2023.

Builds on

49 of 66 references.

References · 49

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,160
  4. 2017ScanNet: Richly-annotated 3D Reconstructions of Indoor Scenescited by 3,088
  5. 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,880
  6. 2019DeepSDF: Learning Continuous Signed Distance Functions for Shape Representationcited by 2,682
  7. 2020Implicit Neural Representations with Periodic Activation Functionscited by 2,214
  8. 2018Occupancy Networks: Learning 3D Reconstruction in Function Spacecited by 2,037
  9. 2020pixelNeRF: Neural Radiance Fields from One or Few Imagescited by 1,574
  10. 2019Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfercited by 1,484
  11. 2017Matterport3D: Learning from RGB-D Data in Indoor Environmentscited by 1,354
  12. 2022TensoRF: Tensorial Radiance Fieldscited by 1,275
37 more
  1. 2016A Point Set Generation Network for 3D Object Reconstruction from a Single Imagecited by 972
  2. 2019PIFu: Pixel-Aligned Implicit Function for High-Resolution Clothed Human Digitizationcited by 818
  3. 2020Accelerating 3D Deep Learning with PyTorch3Dcited by 781
  4. 2020Convolutional Occupancy Networkscited by 760
  5. 2021Depth-supervised NeRF: Fewer Views and Faster Training for Freecited by 732
  6. 2018MegaDepth: Learning Single-View Depth Prediction from Internet Photoscited by 731
  7. 2018Open3D: A Modern Library for 3D Data Processingcited by 717
  8. 2020Hypersim: A Photorealistic Synthetic Dataset for Holistic Indoor Scene Understandingcited by 593
  9. 20163D-R2N2: A Unified Approach for Single and Multi-view 3D Object Reconstructioncited by 587
  10. 2018AtlasNet: A Papier-Mach'e Approach to Learning 3D Surface Generationcited by 539
  11. 2017Ray: A Distributed Framework for Emerging AI Applicationscited by 516
  12. 2014Predicting Depth, Surface Normals and Semantic Labels with a Common Multi-Scale Convolutional Architecturecited by 483
  13. 2016Semantic Scene Completion from a Single Depth Imagecited by 468
  14. 20203D-FRONT: 3D Furnished Rooms with layOuts and semaNTicscited by 375
  15. 2021Omnidata: A Scalable Pipeline for Making Multi-Task Mid-Level Vision Datasets from 3D Scanscited by 336
  16. 2018DeepMVS: Learning Multi-view Stereopsiscited by 277
  17. 2020Neural Unsigned Distance Fields for Implicit Function Learningcited by 265
  18. 2019What Do Single-view 3D Reconstruction Networks Learn?cited by 259
  19. 2021NeuralRecon: Real-Time Coherent 3D Reconstruction from Monocular Videocited by 241
  20. 2018Pix3D: Dataset and Methods for Single-Image 3D Shape Modelingcited by 218
  21. 2016Single-Image Depth Perception in the Wildcited by 179
  22. 2016Learning a Predictable and Generative Vector Representation for Objectscited by 173
  23. 2020Total3DUnderstanding: Joint Layout, Object Pose and Mesh Reconstruction for Indoor Scenes from a Single Imagecited by 170
  24. 2017Learning Efficient Point Cloud Generation for Dense 3D Object Reconstructioncited by 133
  25. 2016BundleFusion: Real-time Globally Consistent 3D Reconstruction using On-the-fly Surface Re-integrationcited by 116
  26. 2018PlaneRCNN: 3D Plane Detection and Reconstruction from a Single Imagecited by 105
  27. 2021Holistic 3D Scene Understanding from a Single Image with Implicit Representationcited by 94
  28. 2016IM2CADcited by 78
  29. 2014Designing Deep Networks for Surface Normal Estimationcited by 59
  30. 2018MVDepthNet: Real-time Multiview Depth Estimation Neural Networkcited by 59
  31. 2019Single-Image Piece-wise Planar 3D Reconstruction via Associative Embeddingcited by 56
  32. 2022GIFS: Neural Implicit Function for General Shape Representationcited by 45
  33. 2017Self-Supervised Intrinsic Image Decompositioncited by 43
  34. 2016Learning Non-Lambertian Object Intrinsics across ShapeNet Categoriescited by 42
  35. 20193D-RelNet: Joint Object and Relational Network for 3D Predictioncited by 28
  36. 2019Deep Reinforcement Learning of Volume-guided Progressive View Inpainting for 3D Point Scene Completion from a Single Depth Imagecited by 15
  37. 2021Recognizing Scenes from Novel Viewpointscited by 5

Built on by

Cited by year

Citing papers per year: 2024: 2.

2 papers.

  1. 2024MVD-Fusion: Single-view 3D via Depth-consistent Multi-view Generationcited by 35
  2. 20243DFIRES: Few Image 3D REconstruction for Scenes with Hidden Surfacecited by 6