mlsanityData to 14 Oct 2026

Multi-View Mesh Reconstruction with Neural Deferred Shading

arXiv 2212.04386 ↗First posted 8 Dec 2022cs.CV cs.GR cs.LGcited by 44

Not on the map. Why?

Abstract (arXiv)

We propose an analysis-by-synthesis method for fast multi-view 3D reconstruction of opaque objects with arbitrary materials and illumination. State-of-the-art methods use both neural surface representations and neural rendering. While flexible, neural surface representations are a significant bottleneck in optimization runtime. Instead, we represent surfaces as triangle meshes and build a differentiable rendering pipeline around triangle rasterization and neural shading. The renderer is used in a gradient descent optimization where both a triangle mesh and a neural shader are jointly optimized to reproduce the multi-view images. We evaluate our method on a public 3D reconstruction dataset and show that it can match the reconstruction accuracy of traditional baselines and neural approaches while surpassing them in optimization runtime. Additionally, we investigate the shader and find that it learns an interpretable representation of appearance, enabling applications such as 3D material editing.

Key facts. Cited by 44 papers in the map (data through 14 Oct 2026). Builds on 26 of its 79 references that are in the map. First posted on arXiv 8 Dec 2022.

Builds on

26 of 79 references.

References · 26

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2019PyTorch: An Imperative Style, High-Performance Deep Learning Librarycited by 20,305
  3. 2019DeepSDF: Learning Continuous Signed Distance Functions for Shape Representationcited by 2,689
  4. 2020Implicit Neural Representations with Periodic Activation Functionscited by 2,238
  5. 2020Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domainscited by 2,200
  6. 2018Occupancy Networks: Learning 3D Reconstruction in Function Spacecited by 2,041
  7. 2021NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstructioncited by 1,712
  8. 2020pixelNeRF: Neural Radiance Fields from One or Few Imagescited by 1,584
  9. 2020Accelerating 3D Deep Learning with PyTorch3Dcited by 784
  10. 2020Convolutional Occupancy Networkscited by 761
  11. 2019Differentiable Volumetric Rendering: Learning Implicit 3D Representations without 3D Supervisioncited by 700
  12. 2021UNISURF: Unifying Neural Implicit Surfaces and Radiance Fields for Multi-View Reconstructioncited by 646
14 more
  1. 2020Modular Primitives for High-Performance Differentiable Renderingcited by 441
  2. 2021Advances in Neural Renderingcited by 365
  3. 2019Deferred Neural Rendering: Image Synthesis using Neural Texturescited by 253
  4. 2021Neural Lumigraph Renderingcited by 105
  5. 2019Pixel2Mesh++: Multi-View 3D Mesh Generation via Deformationcited by 103
  6. 2019Learning to Infer Implicit Surfaces without 3D Supervisioncited by 103
  7. 2020A Survey on Deep Learning Techniques for Stereo-based Depth Estimationcited by 90
  8. 2019Image-based 3D Object Reconstruction: State-of-the-Art and Trends in the Deep Learning Eracited by 87
  9. 2021Unified Shape and SVBRDF Recovery using Differentiable Monte Carlo Renderingcited by 71
  10. 2020Neural Subdivisioncited by 49
  11. 2020Neural Volume Rendering: NeRF And Beyondcited by 44
  12. 2019Photometric Mesh Optimization for Video-Aligned 3D Object Reconstructioncited by 28
  13. 2021Appearance-Driven Automatic 3D Model Simplificationcited by 24
  14. 2020Differentiable Refraction-Tracing for Mesh Reconstruction of Transparent Objectscited by 23

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 14; 2024: 13; 2025: 10; 2026: 7.

44 papers.

Sort:

By area:

  1. 2023Zero-1-to-3: Zero-shot One Image to 3D Objectcited by 1,489
  2. 2023Triplane Meets Gaussian Splatting: Fast and Generalizable Single-View 3D Reconstruction with Transformerscited by 258
  3. 2025SAM 3D: 3Dfy Anything in Imagescited by 247
  4. 2024Unique3D: High-Quality and Efficient 3D Mesh Generation from a Single Imagecited by 135
  5. 2023Multiview Compressive Coding for 3D Reconstructioncited by 71
  6. 2023FLARE: Fast Learning of Animatable and Relightable Mesh Avatarscited by 39
  7. 2024Reconstructing Hand-Held Objects in 3D from Images and Videoscited by 24
  8. 2024TetSphere Splatting: Representing High-Quality Geometry with Lagrangian Volumetric Meshescited by 21
  9. 2023Chupa: Carving 3D Clothed Humans from Skinned Shape Priors using 2D Diffusion Probabilistic Modelscited by 20
  10. 2023Looking Through the Glass: Neural Surface Reconstruction Against High Specular Reflectionscited by 19
  11. 2024GarmentDreamer: 3DGS Guided Garment Synthesis with Diverse Geometry and Texture Detailscited by 19
  12. 2024Tree-D Fusion: Simulation-Ready Tree Dataset from Single Images with Diffusion Priorscited by 13
32 more
  1. 2026Beyond Pixel Histories: World Models with Persistent 3D Statecited by 13
  2. 2024CanonicalFusion: Generating Drivable 3D Human Avatars from Multiple Imagescited by 12
  3. 2024HeadEvolver: Text to Head Avatars via Expressive and Attribute-Preserving Mesh Deformationcited by 11
  4. 2023Explicit Neural Surfaces: Learning Continuous Geometry With Deformation Fieldscited by 10
  5. 2023MVHuman: Tailoring 2D Diffusion with Multi-view Sampling For Realistic 3D Human Generationcited by 6
  6. 2024High-Quality Mesh Blendshape Generation from Face Videos via Neural Inverse Renderingcited by 6
  7. 2024Sur2f: A Hybrid Representation for High-Quality and Efficient Surface Reconstruction from Multi-view Imagescited by 5
  8. 2023Differentiable Shadow Mapping for Efficient Inverse Graphicscited by 4
  9. 2023Sparse 3D Reconstruction via Object-Centric Ray Samplingcited by 4
  10. 2025VaseVQA-3D: Benchmarking 3D VLMs on Ancient Greek Potterycited by 4
  11. 2026DiffSoup: Direct Differentiable Rasterization of Triangle Soup for Extreme Radiance Field Simplificationcited by 4
  12. 2023A Review of Deep Learning-Powered Mesh Reconstruction Methodscited by 3
  13. 2026Fast-SAM3D: 3Dfy Anything in Images but Fastercited by 3
  14. 2023Optimization-Based Eye Tracking using Deflectometric Informationcited by 2
  15. 2023FastMESH: Fast Surface Reconstruction by Hexagonal Mesh-based Neural Renderingcited by 2
  16. 2024REFRAME: Reflective Surface Real-Time Rendering for Mobile Devicescited by 2
  17. 2025DNRSelect: Active Best View Selection for Deferred Neural Renderingcited by 2
  18. 2025A Text-to-3D Framework for Joint Generation of CG-Ready Humans and Compatible Garmentscited by 2
  19. 2025Beyond Reconstruction: A Physics Based Neural Deferred Shader for Photo-realistic Renderingcited by 2
  20. 2025Text-based Animatable 3D Avatars with Morphable Model Alignmentcited by 2
  21. 2023P2CADNet: An End-to-End Reconstruction Network for Parametric 3D CAD Model from Point Cloudscited by 1
  22. 20242L3: Lifting Imperfect Generated 2D Images into Accurate 3Dcited by 1
  23. 20263DCarGen: Scalable 3D Car Generation via 3D-consistent Multi-view Synthesiscited by 1
  24. 2024HVOFusion: Incremental Mesh Reconstruction Using Hybrid Voxel Octreecited by 0
  25. 2024LEMON: Localized Editing with Mesh Optimization and Neural Shaderscited by 0
  26. 20253D Engine-ready Photorealistic Avatars via Dynamic Texturescited by 0
  27. 2025Disentangled Geometry and Appearance for Efficient Multi-View Surface Reconstruction and Renderingcited by 0
  28. 2025VoroLight: Learning Voronoi Surface Meshes via Sphere Intersectioncited by 0
  29. 2025A Convolutional Neural Deferred Shader for Physics Based Renderingcited by 0
  30. 2026CloDS: Visual-Only Unsupervised Cloth Dynamics Learning in Unknown Conditionscited by 0
  31. 2026Convolutional Neural Shading for High-Quality 3D Reconstruction from Multi-View Imagescited by 0
  32. 2026RADmesh: Remesh-Aware Mesh Deformationcited by 0