mlsanityData to 7 Oct 2026

Differentiable Stereopsis: Meshes from multiple views using differentiable rendering

arXiv 2110.05472 ↗First posted 11 Oct 2021cs.CVcited by 26

Not on the map. Why?

Abstract (arXiv)

We propose Differentiable Stereopsis, a multi-view stereo approach that reconstructs shape and texture from few input views and noisy cameras. We pair traditional stereopsis and modern differentiable rendering to build an end-to-end model which predicts textured 3D meshes of objects with varying topologies and shape. We frame stereopsis as an optimization problem and simultaneously update shape and cameras via simple gradient descent. We run an extensive quantitative analysis and compare to traditional multi-view stereo techniques and state-of-the-art learning based methods. We show compelling reconstructions on challenging real-world scenes and for an abundance of object types with complex shape, topology and texture. Project webpage: https://shubham-goel.github.io/ds/

Key facts. Cited by 26 papers in the map (data through 7 Oct 2026). Builds on 22 of its 62 references that are in the map. First posted on arXiv 11 Oct 2021.

Builds on

22 of 62 references.

References · 22

  1. 2014Microsoft COCO: Common Objects in Contextcited by 17,498
  2. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  3. 2016SGDR: Stochastic Gradient Descent with Warm Restartscited by 4,993
  4. 2020Accelerating 3D Deep Learning with PyTorch3Dcited by 781
  5. 2018MVSNet: Depth Inference for Unstructured Multi-view Stereocited by 780
  6. 2017Unsupervised Learning of Depth and Ego-Motion from Videocited by 778
  7. 2018MegaDepth: Learning Single-View Depth Prediction from Internet Photoscited by 731
  8. 2019Differentiable Volumetric Rendering: Learning Implicit 3D Representations without 3D Supervisioncited by 699
  9. 2019PointRend: Image Segmentation as Renderingcited by 439
  10. 2017Neural 3D Mesh Renderercited by 437
  11. 2019Soft Rasterizer: A Differentiable Renderer for Image-based 3D Reasoningcited by 420
  12. 2018GeoNet: Unsupervised Learning of Dense Depth, Optical Flow and Camera Posecited by 341
10 more
  1. 2018Learning Category-Specific Mesh Reconstruction from Image Collectionscited by 317
  2. 2019Learning to Predict 3D Objects with an Interpolation-based Differentiable Renderercited by 221
  3. 2020Self-supervised Single-view 3D Reconstruction via Semantic Consistencycited by 127
  4. 20163D Shape Induction from 2D Views of Multiple Objectscited by 121
  5. 2020Shape and Viewpoint without Keypointscited by 92
  6. 2020Dense Hybrid Recurrent Multi-view Stereo Net with Dynamic Consistency Checkingcited by 81
  7. 2020Articulation-aware Canonical Surface Mappingcited by 80
  8. 2021Shelf-Supervised Mesh Prediction in the Wildcited by 65
  9. 2014Category-Specific Object Reconstruction from a Single Imagecited by 56
  10. 2020Implicit Mesh Reconstruction from Unannotated Image Collectionscited by 49

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 4; 2023: 11; 2024: 7; 2025: 3; 2026: 1.

26 papers.

Sort:

By area:

  1. 2022SparseFusion: Distilling View-conditioned Diffusion for 3D Reconstructioncited by 216
  2. 2024Unique3D: High-Quality and Efficient 3D Mesh Generation from a Single Imagecited by 132
  3. 2024Cameras as Rays: Pose Estimation via Ray Diffusioncited by 129
  4. 2022MagicPony: Learning Articulated 3D Animals in the Wildcited by 111
  5. 2022RelPose: Predicting Probabilistic Relative Rotation for Single Objects in the Wildcited by 102
  6. 2024Meta 3D AssetGen: Text-to-Mesh Generation with High-Quality Geometry, Texture, and PBR Materialscited by 66
  7. 2023Differentiable Blocks World: Qualitative 3D Decomposition by Rendering Primitivescited by 37
  8. 2023Looking Through the Glass: Neural Surface Reconstruction Against High Specular Reflectionscited by 19
  9. 2022SPARF: Large-Scale Learning of 3D Sparse Radiance Fields from Few Input Imagescited by 17
  10. 2023HollowNeRF: Pruning Hashgrid-Based NeRFs with Trainable Collision Mitigationcited by 16
  11. 2023Geometry-biased Transformers for Novel View Synthesiscited by 9
  12. 2024GLS: Geometry-aware 3D Language Gaussian Splattingcited by 9
14 more
  1. 2023TMO: Textured Mesh Acquisition of Objects with a Mobile Device by using Differentiable Renderingcited by 7
  2. 2024Sur2f: A Hybrid Representation for High-Quality and Efficient Surface Reconstruction from Multi-view Imagescited by 5
  3. 2024The More You See in 2D, the More You Perceive in 3Dcited by 5
  4. 2023Sparse 3D Reconstruction via Object-Centric Ray Samplingcited by 4
  5. 2023A Review of Deep Learning-Powered Mesh Reconstruction Methodscited by 3
  6. 2023GeoScaler: Geometry and Rendering-Aware Downsampling of 3D Mesh Texturescited by 3
  7. 2024Meta 3D Gencited by 3
  8. 2025IM360: Large-scale Indoor Mapping with 360 Camerascited by 3
  9. 20233D Surface Reconstruction in the Wild by Deforming Shape Priors from Synthetic Datacited by 2
  10. 2023FastMESH: Fast Surface Reconstruction by Hexagonal Mesh-based Neural Renderingcited by 2
  11. 2023Partial-View Object View Synthesis via Filtered Inversioncited by 1
  12. 2025Disentangled Geometry and Appearance for Efficient Multi-View Surface Reconstruction and Renderingcited by 0
  13. 2025ProFusion: 3D Reconstruction of Protein Complex Structures from Multi-view AFM Imagescited by 0
  14. 2026Category-Level 3D Correspondence in Camera Space via Morphable Object Priorscited by 0