mlsanityData to 7 Oct 2026

Black-Box Test-Time Shape REFINEment for Single View 3D Reconstruction

arXiv 2108.09911 ↗First posted 23 Aug 2021cs.CVcited by 3

Not on the map. Why?

Abstract (arXiv)

Much recent progress has been made in reconstructing the 3D shape of an object from an image of it, i.e. single view 3D reconstruction. However, it has been suggested that current methods simply adopt a "nearest-neighbor" strategy, instead of genuinely understanding the shape behind the input image. In this paper, we rigorously show that for many state of the art methods, this issue manifests as (1) inconsistencies between coarse reconstructions and input images, and (2) inability to generalize across domains. We thus propose REFINE, a postprocessing mesh refinement step that can be easily integrated into the pipeline of any black-box method in the literature. At test time, REFINE optimizes a network per mesh instance, to encourage consistency between the mesh and the given object view. This, along with a novel combination of regularizing losses, reduces the domain gap and achieves state of the art performance. We believe that this novel paradigm is an important step towards robust, accurate reconstructions, remaining relevant as new reconstruction networks are introduced.

Key facts. Cited by 3 papers in the map (data through 7 Oct 2026). Builds on 52 of its 78 references that are in the map. First posted on arXiv 23 Aug 2021.

Builds on

52 of 78 references.

References · 52

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2016Semi-Supervised Classification with Graph Convolutional Networkscited by 11,282
  3. 2017Mask R-CNNcited by 7,259
  4. 2015Domain-Adversarial Training of Neural Networkscited by 3,023
  5. 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,880
  6. 2019DeepSDF: Learning Continuous Signed Distance Functions for Shape Representationcited by 2,682
  7. 2018Occupancy Networks: Learning 3D Reconstruction in Function Spacecited by 2,037
  8. 2015SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentationcited by 1,483
  9. 2017Adversarial Discriminative Domain Adaptationcited by 1,309
  10. 2016Deep CORAL: Correlation Alignment for Deep Domain Adaptationcited by 1,194
  11. 2017Deep Image Priorcited by 1,097
  12. 2016A Point Set Generation Network for 3D Object Reconstruction from a Single Imagecited by 972
38 more
  1. 2020Accelerating 3D Deep Learning with PyTorch3Dcited by 781
  2. 2017PoseCNN: A Convolutional Neural Network for 6D Object Pose Estimation in Cluttered Scenescited by 768
  3. 2019Differentiable Volumetric Rendering: Learning Implicit 3D Representations without 3D Supervisioncited by 699
  4. 2018Pixel2Mesh: Generating 3D Mesh Models from Single RGB Imagescited by 649
  5. 2016Deep Transfer Learning with Joint Adaptation Networkscited by 609
  6. 20163D-R2N2: A Unified Approach for Single and Multi-view 3D Object Reconstructioncited by 587
  7. 2018AtlasNet: A Papier-Mach'e Approach to Learning 3D Surface Generationcited by 539
  8. 2017Neural 3D Mesh Renderercited by 437
  9. 2019Soft Rasterizer: A Differentiable Renderer for Image-based 3D Reasoningcited by 420
  10. 2019DISN: Deep Implicit Surface Network for High-quality Single-view 3D Reconstructioncited by 332
  11. 2018Learning Category-Specific Mesh Reconstruction from Image Collectionscited by 317
  12. 2019What Do Single-view 3D Reconstruction Networks Learn?cited by 259
  13. 2018Pix3D: Dataset and Methods for Single-Image 3D Shape Modelingcited by 218
  14. 2017T-LESS: An RGB-D Dataset for 6D Pose Estimation of Texture-less Objectscited by 210
  15. 2017Real-Time Seamless Single Shot 6D Object Pose Predictioncited by 210
  16. 2017Octree Generating Networks: Efficient Convolutional Architectures for High-resolution 3D Outputscited by 189
  17. 2019Unsupervised Learning of Probably Symmetric Deformable 3D Objects from Images in the Wildcited by 180
  18. 2019Pix2Vox: Context-aware 3D Reconstruction from Single and Multi-view Imagescited by 164
  19. 2017Learning Efficient Point Cloud Generation for Dense 3D Object Reconstructioncited by 133
  20. 2020Self-supervised Single-view 3D Reconstruction via Semantic Consistencycited by 127
  21. 2020f-BRS: Rethinking Backpropagating Refinement for Interactive Segmentationcited by 117
  22. 2020MeshSDF: Differentiable Iso-Surface Extractioncited by 112
  23. 2017MarrNet: 3D Shape Reconstruction via 2.5D Sketchescited by 108
  24. 2019Three-D Safari: Learning to Estimate Zebra Pose, Shape, and Texture from Images "In the Wild"cited by 106
  25. 2018Segmentation-driven 6D Object Pose Estimationcited by 104
  26. 2017Self-supervised Learning of Motion Capturecited by 97
  27. 2016A Large Dataset of Object Scanscited by 64
  28. 2020Tent: Fully Test-time Adaptation by Entropy Minimizationcited by 59
  29. 2020Online Adaptation for Consistent Mesh Reconstruction in the Wildcited by 51
  30. 2017O-CNN: Octree-based Convolutional Neural Networks for 3D Shape Analysiscited by 47
  31. 2020Point2Mesh: A Self-Prior for Deformable Meshescited by 41
  32. 2020ManifoldPlus: A Robust and Scalable Watertight Manifold Surface Generation Method for Triangle Soupscited by 38
  33. 2019Interactive segmentation of medical images through fully convolutional neural networkscited by 29
  34. 2019Photometric Mesh Optimization for Video-Aligned 3D Object Reconstructioncited by 28
  35. 2019Few-Shot Generalization for Single-Image 3D Reconstruction via Priorscited by 25
  36. 2019PRS-Net: Planar Reflective Symmetry Detection Net for 3D Modelscited by 20
  37. 2021NeRD: Neural 3D Reflection Symmetry Detectorcited by 18
  38. 2018Domain-Adaptive Single-View 3D Reconstructioncited by 12

50 most cited of 52 All 52 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 1; 2023: 1; 2024: 0; 2025: 0; 2026: 1.

3 papers.

  1. 2023A Comprehensive Survey on Test-Time Adaptation under Distribution Shiftscited by 353
  2. 2022DISCO: Adversarial Defense with Local Implicit Functionscited by 33
  3. 2026ASV3D: Adapting Diffusion-Based Single-View 3D Reconstruction with Extra Imagerycited by 1