mlsanityData to 7 Oct 2026

Structure from Duplicates: Neural Inverse Graphics from a Pile of Objects

arXiv 2401.05236 ↗First posted 10 Jan 2024cs.CVcited by 5

Not on the map. Why?

Abstract (arXiv)

Our world is full of identical objects (emphe.g., cans of coke, cars of same model). These duplicates, when seen together, provide additional and strong cues for us to effectively reason about 3D. Inspired by this observation, we introduce Structure from Duplicates (SfD), a novel inverse graphics framework that reconstructs geometry, material, and illumination from a single image containing multiple identical objects. SfD begins by identifying multiple instances of an object within an image, and then jointly estimates the 6DoF pose for all instances.An inverse graphics pipeline is subsequently employed to jointly reason about the shape, material of the object, and the environment light, while adhering to the shared geometry and material constraint across instances. Our primary contributions involve utilizing object duplicates as a robust prior for single-image inverse graphics and proposing an in-plane rotation-robust Structure from Motion (SfM) formulation for joint 6-DoF object pose estimation. By leveraging multi-view cues from a single image, SfD generates more realistic and detailed 3D reconstructions, significantly outperforming existing single image reconstruction models and multi-view reconstruction approaches with a similar or greater number of observations.

Key facts. Cited by 5 papers in the map (data through 7 Oct 2026). Builds on 43 of its 74 references that are in the map. First posted on arXiv 10 Jan 2024.

Builds on

43 of 74 references.

References · 43

  1. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  2. 2020Implicit Neural Representations with Periodic Activation Functionscited by 2,214
  3. 2021Vision Transformers for Dense Predictioncited by 1,789
  4. 2021NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstructioncited by 1,701
  5. 2019Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfercited by 1,484
  6. 2020D-NeRF: Neural Radiance Fields for Dynamic Scenescited by 1,459
  7. 2019SuperGlue: Learning Feature Matching with Graph Neural Networkscited by 1,437
  8. 2017SuperPoint: Self-Supervised Interest Point Detection and Descriptioncited by 1,313
  9. 2014Depth Map Prediction from a Single Image using a Multi-Scale Deep Networkcited by 1,264
  10. 2023Fantasia3D: Disentangling Geometry and Appearance for High-quality Text-to-3D Content Creationcited by 684
  11. 2021BARF: Bundle-Adjusting Neural Radiance Fieldscited by 641
  12. 2020NeRV: Neural Reflectance and Visibility Fields for Relighting and View Synthesiscited by 493
31 more
  1. 2023Neuralangelo: High-Fidelity Neural Surface Reconstructioncited by 464
  2. 2021Extracting Triangular 3D Models, Materials, and Lighting From Imagescited by 363
  3. 2021Omnidata: A Scalable Pipeline for Making Multi-Task Mid-Level Vision Datasets from 3D Scanscited by 336
  4. 2021PhySG: Inverse Rendering with Spherical Gaussians for Physics-based Material Editing and Relightingcited by 332
  5. 2023UniSim: A Neural Closed-Loop Sensor Simulatorcited by 294
  6. 2021Editing Conditional Radiance Fieldscited by 222
  7. 2022SPARF: Neural Radiance Fields from Sparse and Noisy Posescited by 195
  8. 2016Single-Image Depth Perception in the Wildcited by 179
  9. 2021Mask2Former for Video Instance Segmentationcited by 169
  10. 2022Modeling Indirect Illumination for Inverse Renderingcited by 167
  11. 2022Shape, Light, and Material Decomposition from Images using Monte Carlo Rendering and Denoisingcited by 163
  12. 2021Dex-NeRF: Using a Neural Radiance Field to Grasp Transparent Objectscited by 119
  13. 2022IRON: Inverse Rendering by Optimizing Neural SDFs and Materials from Photometric Imagescited by 110
  14. 2020Image GANs meet Differentiable Rendering for Inverse Graphics and Interpretable 3D Neural Renderingcited by 101
  15. 2023Neural Fields meet Explicit Geometric Representation for Inverse Rendering of Urban Scenescited by 89
  16. 2015Recovering 6D Object Pose and Predicting Next-Best-View in the Crowdcited by 79
  17. 2023CamP: Camera Preconditioning for Neural Radiance Fieldscited by 55
  18. 2021DIB-R++: Learning to Predict Lighting and Material with a Hybrid Differentiable Renderercited by 52
  19. 2019GIFT: Learning Transformation-Invariant Dense Visual Descriptors via Group CNNscited by 47
  20. 2021Reconstructing 3D Human Pose by Watching Humans in the Mirrorcited by 47
  21. 2023PET-NeuS: Positional Encoding Tri-Planes for Neural Surfacescited by 37
  22. 2021Shape and Material Capture at Homecited by 36
  23. 2022MIRA: Mental Imagery for Robotic Affordancescited by 26
  24. 2023CADSim: Robust and Scalable in-the-wild 3D Reconstruction for Controllable Sensor Simulationcited by 26
  25. 2015Learning Data-driven Reflectance Priors for Intrinsic Image Decompositioncited by 24
  26. 2019Exploiting Sparse Semantic HD Maps for Self-Driving Vehicle Localizationcited by 24
  27. 2022PEANUT: Predicting and Navigating to Unseen Targetscited by 22
  28. 2022Seeing a Rose in Five Thousand Wayscited by 13
  29. 2020Multi-Plane Program Induction with 3D Box Priorscited by 10
  30. 2020Perspective Plane Program Induction from a Single Imagecited by 9
  31. 2019Scene-level Pose Estimation for Multiple Instances of Densely Packed Objectscited by 8

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 1; 2025: 2; 2026: 2.

5 papers.

  1. 2024DreamMat: High-quality PBR Material Generation with Geometry- and Light-aware Diffusion Modelscited by 50
  2. 2025DNF-Intrinsic: Deterministic Noise-Free Diffusion for Indoor Inverse Renderingcited by 11
  3. 2025Splat and Replace: 3D Reconstruction with Repetitive Elementscited by 4
  4. 2026FIRE3D: Feed-forward Interactive 3D Scene Reconstruction Within A Minutecited by 3
  5. 2026Bottom-up Modeling of Repeated Elements via Single Image Analysis-by-Synthesiscited by 0