mlsanityData to 14 Oct 2026

Shadows Shed Light on 3D Objects

arXiv 2206.08990 ↗First posted 17 Jun 2022cs.CV cs.GRcited by 8

Not on the map. Why?

Abstract (arXiv)

3D reconstruction is a fundamental problem in computer vision, and the task is especially challenging when the object to reconstruct is partially or fully occluded. We introduce a method that uses the shadows cast by an unobserved object in order to infer the possible 3D volumes behind the occlusion. We create a differentiable image formation model that allows us to jointly infer the 3D shape of an object, its pose, and the position of a light source. Since the approach is end-to-end differentiable, we are able to integrate learned priors of object geometry in order to generate realistic 3D shapes of different object categories. Experiments and visualizations show that the method is able to generate multiple possible solutions that are consistent with the observation of the shadow. Our approach works even when the position of the light source and object pose are both unknown. Our approach is also robust to real-world images where ground-truth shadow mask is unknown.

Key facts. Cited by 8 papers in the map (data through 14 Oct 2026). Builds on 22 of its 44 references that are in the map. First posted on arXiv 17 Jun 2022.

Builds on

22 of 44 references.

References · 22

  1. 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,897
  2. 2018Occupancy Networks: Learning 3D Reconstruction in Function Spacecited by 2,041
  3. 2020pixelNeRF: Neural Radiance Fields from One or Few Imagescited by 1,584
  4. 2014Image Super-Resolution Using Deep Convolutional Networkscited by 1,224
  5. 2016A Point Set Generation Network for 3D Object Reconstruction from a Single Imagecited by 979
  6. 2015Accurate Image Super-Resolution Using Very Deep Convolutional Networkscited by 903
  7. 2019Differentiable Volumetric Rendering: Learning Implicit 3D Representations without 3D Supervisioncited by 700
  8. 2018AtlasNet: A Papier-Mach'e Approach to Learning 3D Surface Generationcited by 539
  9. 2019Soft Rasterizer: A Differentiable Renderer for Image-based 3D Reasoningcited by 420
  10. 2020PULSE: Self-Supervised Photo Upsampling via Latent Space Exploration of Generative Modelscited by 358
  11. 2018Learning Category-Specific Mesh Reconstruction from Image Collectionscited by 317
  12. 2018Learning 3D Human Dynamics from Videocited by 316
10 more
  1. 2020Deep Learning Techniques for Inverse Problems in Imagingcited by 291
  2. 2019Unsupervised Learning of Probably Symmetric Deformable 3D Objects from Images in the Wildcited by 180
  3. 2016Generative and Discriminative Voxel Modeling with Convolutional Neural Networkscited by 128
  4. 2020Self-supervised Single-view 3D Reconstruction via Semantic Consistencycited by 127
  5. 2021Shelf-Supervised Mesh Prediction in the Wildcited by 65
  6. 2019Instance Shadow Detectioncited by 37
  7. 2015Learning Analysis-by-Synthesis for 6D Pose Estimation in RGB-D Imagescited by 29
  8. 2017Image2Mesh: A Learning Framework for Single Image 3D Reconstructioncited by 29
  9. 2018Pixels, voxels, and views: A study of shape representations for single view 3D object shape predictioncited by 27
  10. 2021Shadow Art Revisited: A Differentiable Rendering Based Approachcited by 10

Built on by

Cited by year

Citing papers per year: 2022: 2; 2023: 5; 2024: 1.

8 papers.

  1. 2023Zero-1-to-3: Zero-shot One Image to 3D Objectcited by 1,489
  2. 2023Objaverse-XL: A Universe of 10M+ 3D Objectscited by 774
  3. 2024pix2gestalt: Amodal Segmentation by Synthesizing Wholescited by 70
  4. 2022FLEX: Full-Body Grasping Without Full-Body Graspscited by 46
  5. 2023Sin3DM: Learning a Diffusion Model from a Single 3D Textured Shapecited by 29
  6. 2023Humans as Light Bulbs: 3D Human Reconstruction from Thermal Reflectioncited by 10
  7. 2023PlatoNeRF: 3D Reconstruction in Plato's Cave via Single-View Two-Bounce Lidarcited by 9
  8. 2022Probabilistic Shape Completion by Estimating Canonical Factors with Hierarchical VAEcited by 1