mlsanityData to 7 Oct 2026

Shelf-Supervised Mesh Prediction in the Wild

arXiv 2102.06195 ↗First posted 11 Feb 2021cs.CVcited by 65

Not on the map. Why?

Abstract (arXiv)

We aim to infer 3D shape and pose of object from a single image and propose a learning-based approach that can train from unstructured image collections, supervised by only segmentation outputs from off-the-shelf recognition systems (i.e. 'shelf-supervised'). We first infer a volumetric representation in a canonical frame, along with the camera pose. We enforce the representation geometrically consistent with both appearance and masks, and also that the synthesized novel views are indistinguishable from image collections. The coarse volumetric prediction is then converted to a mesh-based representation, which is further refined in the predicted camera frame. These two steps allow both shape-pose factorization from image collections and per-instance reconstruction in finer details. We examine the method on both synthetic and real-world datasets and demonstrate its scalability on 50 categories in the wild, an order of magnitude more classes than existing works.

Key facts. Cited by 65 papers in the map (data through 7 Oct 2026). Builds on 43 of its 57 references that are in the map. First posted on arXiv 11 Feb 2021.

Builds on

43 of 57 references.

References · 43

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  3. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  4. 2018A Style-Based Generator Architecture for Generative Adversarial Networkscited by 5,986
  5. 2016DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFscited by 3,239
  6. 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,880
  7. 2019DeepSDF: Learning Continuous Signed Distance Functions for Shape Representationcited by 2,682
  8. 2019Scene Representation Networks: Continuous 3D-Structure-Aware Neural Scene Representationscited by 864
  9. 2017Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networkscited by 707
  10. 2016Keep it SMPL: Automatic Estimation of 3D Human Pose and Shape from a Single Imagecited by 692
  11. 2018Pixel2Mesh: Generating 3D Mesh Models from Single RGB Imagescited by 649
  12. 20163D-R2N2: A Unified Approach for Single and Multi-view 3D Object Reconstructioncited by 587
31 more
  1. 2016Learning a Probabilistic Latent Space of Object Shapes via 3D Generative-Adversarial Modelingcited by 558
  2. 2015Deep Metric Learning via Lifted Structured Feature Embeddingcited by 449
  3. 2019PointRend: Image Segmentation as Renderingcited by 439
  4. 2019Soft Rasterizer: A Differentiable Renderer for Image-based 3D Reasoningcited by 420
  5. 2015Autoencoding beyond pixels using a learned similarity metriccited by 407
  6. 2018DeepVoxels: Learning Persistent 3D Feature Embeddingscited by 401
  7. 2018Learning Category-Specific Mesh Reconstruction from Image Collectionscited by 317
  8. 2020State of the Art on Neural Renderingcited by 276
  9. 2019What Do Single-view 3D Reconstruction Networks Learn?cited by 259
  10. 2019Unsupervised Learning of Probably Symmetric Deformable 3D Objects from Images in the Wildcited by 180
  11. 2017Multi-view Supervision for Single-view Reconstruction via Differentiable Ray Consistencycited by 177
  12. 2016Learning a Predictable and Generative Vector Representation for Objectscited by 173
  13. 2018Escaping Plato's Cave: 3D Shape From Adversarial Renderingcited by 170
  14. 2018Monocular Total Capture: Posing Face, Body, and Hands in the Wildcited by 162
  15. 2020Self-supervised Single-view 3D Reconstruction via Semantic Consistencycited by 127
  16. 20163D Shape Induction from 2D Views of Multiple Objectscited by 121
  17. 2019Neural Rerendering in the Wildcited by 121
  18. 2019Large-scale interactive object segmentation with human annotatorscited by 109
  19. 2016Perspective Transformer Nets: Learning Single-View 3D Object Reconstruction without 3D Supervisioncited by 106
  20. 2019Learning to Infer Implicit Surfaces without 3D Supervisioncited by 103
  21. 2015Render for CNN: Viewpoint Estimation in Images Using CNNs Trained with Rendered 3D Model Viewscited by 94
  22. 2018LiveCap: Real-time Human Performance Capture from Monocular Videocited by 93
  23. 2020Shape and Viewpoint without Keypointscited by 92
  24. 2020Articulation-aware Canonical Surface Mappingcited by 80
  25. 2020Leveraging 2D Data to Learn Textured 3D Mesh Generationcited by 69
  26. 2019Canonical Surface Mapping via Geometric Cycle Consistencycited by 62
  27. 2019Learning single-image 3D reconstruction by generative modelling of shape, pose and shadingcited by 43
  28. 2019Deep Non-Rigid Structure from Motioncited by 38
  29. 2018Learning View Priors for Single-view 3D Reconstructioncited by 35
  30. 2019GEOMetrics: Exploiting Geometric Structure for Graph-Encoded Objectscited by 27
  31. 2017SilNet : Single- and Multi-View Reconstruction by Learning from Silhouettescited by 24

Built on by

Cited by year

Citing papers per year: 2021: 5; 2022: 26; 2023: 21; 2024: 9; 2025: 4.

65 papers.

Sort:

By area:

  1. 2023RealFusion: 360deg Reconstruction of Any Object from a Single Imagecited by 305
  2. 2022SparseFusion: Distilling View-conditioned Diffusion for 3D Reconstructioncited by 216
  3. 2021BANMo: Building Animatable 3D Neural Models from Many Casual Videoscited by 195
  4. 2022NeRDi: Single-View NeRF Synthesis with Language-Guided Diffusion as General Image Priorscited by 156
  5. 2022EpiGRAF: Rethinking training of 3D GANscited by 108
  6. 2022What's in your hands? 3D Reconstruction of Generic Objects in Handscited by 90
  7. 2022LASSIE: Learning Articulated Shapes from Sparse Image Ensemble via 3D Part Discoverycited by 65
  8. 2021DOVE: Learning Deformable 3D Objects by Watching Videoscited by 61
  9. 2024Learning the 3D Fauna of the Webcited by 58
  10. 2022Few-View Object Reconstruction with Unknown Categories and Camera Posescited by 53
  11. 2021FiG-NeRF: Figure-Ground Neural Radiance Fields for 3D Object Category Modellingcited by 47
  12. 2023Reconstructing Animatable Categories from Videoscited by 46
38 more
  1. 2024REACTO: Reconstructing Articulated Objects from a Single Videocited by 45
  2. 2024MVD-Fusion: Single-view 3D via Depth-consistent Multi-view Generationcited by 35
  3. 2022De-rendering 3D Objects in the Wildcited by 31
  4. 2022Hi-LASSIE: High-Fidelity Articulated Shape and Skeleton Discovery from Sparse Image Ensemblecited by 30
  5. 2022Learning Dexterous Manipulation from Exemplar Object Trajectories and Pre-Graspscited by 27
  6. 2021Differentiable Stereopsis: Meshes from multiple views using differentiable renderingcited by 26
  7. 2022Pre-train, Self-train, Distill: A simple recipe for Supersizing 3D Reconstructioncited by 26
  8. 2022Share With Thy Neighbors: Single-View Reconstruction by Cross-Instance Consistencycited by 26
  9. 2024Real3D: Scaling Up Large Reconstruction Models with Real-World Imagescited by 24
  10. 2025DSO: Aligning 3D Generators with Simulation Feedback for Physical Soundnesscited by 24
  11. 2022Self-Supervised Geometric Correspondence for Category-Level 6D Object Pose Estimation in the Wildcited by 22
  12. 2023ARTIC3D: Learning Robust Articulated 3D Shapes from Noisy Web Image Collectionscited by 22
  13. 2025PICO: Reconstructing 3D People In Contact with Objectscited by 22
  14. 2022Pixel2Mesh++: 3D Mesh Generation and Refinement from Multi-View Imagescited by 21
  15. 2022Learning 3D Object Shape and Layout without 3D Supervisioncited by 19
  16. 2023ShapeClipper: Scalable 3D Shape Learning from Single-View Images via Geometric and CLIP-based Consistencycited by 16
  17. 2023DDF-HO: Hand-Held Object Reconstruction via Conditional Directed Distance Fieldcited by 16
  18. 2023SAOR: Single-View Articulated Object Reconstructioncited by 15
  19. 2023Sketch-A-Shape: Zero-Shot Sketch-to-3D Shape Generationcited by 14
  20. 2023UpFusion: Novel View Diffusion from Unposed Sparse View Observationscited by 14
  21. 2023MoDA: Modeling Deformable 3D Objects from Casual Videoscited by 13
  22. 2022Towards Learning Neural Representations from Shadowscited by 12
  23. 20233D Reconstruction of Objects in Hands without Real World 3D Supervisioncited by 12
  24. 2023DreaMo: Articulated 3D Reconstruction From A Single Casual Videocited by 12
  25. 2024DressRecon: Freeform 4D Human Reconstruction from Monocular Videocited by 12
  26. 20222D GANs Meet Unsupervised Single-view 3D Reconstructioncited by 11
  27. 2022im2nerf: Image to Neural Radiance Field in the Wildcited by 11
  28. 2022Single-view 3D Mesh Reconstruction for Seen and Unseen Categoriescited by 10
  29. 2023CAMM: Building Category-Agnostic and Animatable 3D Models from Monocular Videoscited by 10
  30. 2023Humans as Light Bulbs: 3D Human Reconstruction from Thermal Reflectioncited by 10
  31. 2022Planes vs. Chairs: Category-guided 3D shape learning without any 3D cuescited by 9
  32. 2024SDFit: 3D Object Pose and Shape by Fitting a Morphable SDF to a Single Imagecited by 9
  33. 2022Shadows Shed Light on 3D Objectscited by 8
  34. 2022GAN2X: Non-Lambertian Inverse Rendering of Image GANscited by 8
  35. 2022Monocular 3D Object Reconstruction with GAN Inversioncited by 8
  36. 2023Root Pose Decomposition Towards Generic Non-rigid 3D Reconstruction with Monocular Videoscited by 8
  37. 2024BiTT: Bi-directional Texture Reconstruction of Interacting Two Hands from a Single Imagecited by 8
  38. 2022Physics vs. Learned Priors: Rethinking Camera and Algorithm Design for Task-Specific Imagingcited by 6

50 most cited of 65 All 65 (JSON)