SDFDiff: Differentiable Rendering of Signed Distance Fields for 3D Shape Optimization
Not on the map. Why?
Abstract (arXiv)
We propose SDFDiff, a novel approach for image-based shape optimization using differentiable rendering of 3D shapes represented by signed distance functions (SDFs). Compared to other representations, SDFs have the advantage that they can represent shapes with arbitrary topology, and that they guarantee watertight surfaces. We apply our approach to the problem of multi-view 3D reconstruction, where we achieve high reconstruction quality and can capture complex topology of 3D objects. In addition, we employ a multi-resolution strategy to obtain a robust optimization algorithm. We further demonstrate that our SDF-based differentiable renderer can be integrated with deep learning models, which opens up options for learning approaches on 3D objects without 3D supervision. In particular, we apply our method to single-view 3D reconstruction and achieve state-of-the-art results.
Key facts. Cited by 159 papers in the map (data through 14 Oct 2026). First posted on arXiv 15 Dec 2019.
Builds on
No reference list yet.
Built on by
Citing papers per year: 2021: 38; 2022: 42; 2023: 45; 2024: 20; 2025: 11; 2026: 3.
159 papers.
Sort:
By area:
- 2021NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstructioncited by 1,712
- 2021Efficient Geometry-aware 3D Generative Adversarial Networkscited by 1,205
- 2021IBRNet: Learning Multi-View Image-Based Renderingcited by 703
- 2021RegNeRF: Regularizing Neural Radiance Fields for View Synthesis from Sparse Inputscited by 615
- 2022MonoSDF: Exploring Monocular Geometric Cues for Neural Implicit Surface Reconstructioncited by 470
- 2023Neuralangelo: High-Fidelity Neural Surface Reconstructioncited by 467
- 2021Neural Geometric Level of Detail: Real-time Rendering with Implicit 3D Shapescited by 426
- 2021Extracting Triangular 3D Models, Materials, and Lighting From Imagescited by 366
- 2021Advances in Neural Renderingcited by 365
- 2021PhySG: Inverse Rendering with Spherical Gaussians for Physics-based Material Editing and Relightingcited by 335
- 2023OmniObject3D: Large-Vocabulary 3D Object Dataset for Realistic Perception, Reconstruction and Generationcited by 326
- 2022NeuS2: Fast Learning of Neural Implicit Surfaces for Multi-view Reconstructioncited by 314
38 more
- 2022SDFusion: Multimodal 3D Shape Completion, Reconstruction, and Generationcited by 288
- 20223D Neural Field Generation using Triplane Diffusioncited by 274
- 2021HeadNeRF: A Real-time NeRF-based Parametric Head Modelcited by 228
- 2022AutoSDF: Shape Priors for 3D Completion, Reconstruction and Generationcited by 208
- 2022SparseNeuS: Fast Generalizable Neural Surface Reconstruction from Sparse Viewscited by 177
- 2021SnowflakeNet: Point Cloud Completion by Snowflake Point Deconvolution with Skip-Transformercited by 164
- 2022Dream3D: Zero-Shot Text-to-3D Synthesis Using 3D Shape Prior and Text-to-Image Diffusion Modelscited by 151
- 2021Efficient Neural Radiance Fields for Interactive Free-viewpoint Videocited by 150
- 2022SelfRecon: Self Reconstruction Your Digital Avatar from Monocular Videocited by 144
- 2021BACON: Band-limited Coordinate Networks for Multiscale Scene Representationcited by 143
- 2021Real-time Deep Dynamic Characterscited by 133
- 2022Diffusion-SDF: Text-to-Shape via Voxelized Diffusioncited by 128
- 2022iSDF: Real-Time Neural Signed Distance Fields for Robot Perceptioncited by 109
- 2021Neural Lumigraph Renderingcited by 105
- 2022Generative Neural Articulated Radiance Fieldscited by 101
- 2022Voxurf: Voxel-based Efficient and Accurate Neural Surface Reconstructioncited by 94
- 20234K4D: Real-Time 4D View Synthesis at 4K Resolutioncited by 91
- 20243DGSR: Implicit Surface Reconstruction with 3D Gaussian Splattingcited by 79
- 2021DiGS : Divergence guided shape implicit neural representation for unoriented point cloudscited by 77
- 2022PermutoSDF: Fast Multi-View Reconstruction with Implicit Surfaces using Permutohedral Latticescited by 73
- 2021Unified Shape and SVBRDF Recovery using Differentiable Monte Carlo Renderingcited by 71
- 2022Unified Implicit Neural Stylizationcited by 67
- 2021ACORN: Adaptive Coordinate Networks for Neural Scene Representationcited by 65
- 2021Compressive Neural Representations of Volumetric Scalar Fieldscited by 64
- 2022NeuralUDF: Learning Unsigned Distance Fields for Multi-view Reconstruction of Surfaces with Arbitrary Topologiescited by 63
- 2021CAPRI-Net: Learning Compact CAD Shapes with Adaptive Primitive Assemblycited by 62
- 2022Reconstructing Surfaces for Sparse Point Clouds with On-Surface Priorscited by 60
- 2022PMP-Net++: Point Cloud Completion by Transformer-Enhanced Multi-step Point Moving Pathscited by 57
- 2023Im4D: High-Fidelity and Real-Time Novel View Synthesis for Dynamic Scenescited by 55
- 2022Surface Reconstruction from Point Clouds by Learning Predictive Context Priorscited by 54
- 2023Cross-Ray Neural Radiance Fields for Novel-view Synthesis from Unconstrained Image Collectionscited by 54
- 2021DoubleField: Bridging the Neural Surface and Radiance Fields for High-fidelity Human Reconstruction and Renderingcited by 52
- 2021Generative Occupancy Fields for 3D Surface-Aware Image Synthesiscited by 51
- 2021Cycle4Completion: Unpaired Point Cloud Completion using Cycle Transformation with Missing Region Codingcited by 50
- 2022CAP-UDF: Learning Unsigned Distance Functions Progressively from Raw Point Clouds with Consistency-Aware Field Optimizationcited by 47
- 2024When LLMs step into the 3D World: A Survey and Meta-Analysis of 3D Tasks via Multi-modal Large Language Modelscited by 43
- 2022A Level Set Theory for Neural Implicit Evolution under Explicit Flowscited by 42
- 2023MixNeRF: Modeling a Ray with Mixture Density for Novel View Synthesis from Sparse Inputscited by 40
50 most cited of 159 All 159 (JSON)
- 2026Differentiable Voxelization of Surface Representations16 Aug 2026
- 2026Stochastic Signed Distance Processes18 Jun 2026
- 2026S2MDF: A Plug-And-Play Layer for Intersection-Free Multi-Object Signed Distance Fields28 May 2026
- 2025Inverse Rendering for High-Genus Surface Meshes from Multi-View Images24 Nov 2025
- 2025Neural Geometry Image-Based Representations with Optimal Transport (OT)24 Nov 2025
- 2025LAMP: Data-Efficient Linear Affine Weight-Space Models for Parameter-Controlled 3D Shape Generation and Extrapolation26 Oct 2025
- 2025MV-Performer: Taming Video Diffusion Model for Faithful and Synchronized Multi-view Performer Synthesis8 Oct 2025
- 2025Efficient multi-view training for 3D Gaussian Splatting15 Jun 2025
- 2025Learning to Upscale 3D Segmentations in Neuroimaging27 May 2025
- 2025Sparfels: Fast Reconstruction from Sparse Unposed Imagery4 May 2025
- 2025SparSplat: Fast Multi-View Reconstruction with Generalizable 2D Gaussian Splatting4 May 2025
- 2025Learning Scene-Level Signed Directional Distance Function with Ellipsoidal Priors and Neural Residuals25 Mar 2025
38 more
- 2025Glissando-Net: Deep sinGLe vIew category level poSe eStimation ANd 3D recOnstruction24 Jan 2025
- 2025Surface-SOS: Self-Supervised Object Segmentation via Neural Surface Representation17 Jan 2025
- 2024Sensing Surface Patches in Volume Rendering for Inferring Signed Distance Functions21 Dec 2024
- 2024HotSpot: Signed Distance Function Optimization with an Asymptotically Sufficient Condition21 Nov 2024
- 2024Inferring Neural Signed Distance Functions by Overfitting on Single Noisy Point Clouds through Finetuning Data-Driven based Priors25 Oct 2024
- 2024Toward General Object-level Mapping from Sparse Views with 3D Diffusion Priors7 Oct 2024
- 2024GeoTransfer : Generalizable Few-Shot Multi-View Reconstruction via Transfer Learning27 Aug 2024
- 2024Learning Local Pattern Modularization for Point Cloud Reconstruction from Unseen Classes26 Aug 2024
- 2024AdaOcc: Adaptive-Resolution Occupancy Prediction24 Aug 2024
- 2024VF-NeRF: Learning Neural Vector Fields for Indoor Scene Reconstruction16 Aug 2024
- 2024BoostMVSNeRFs: Boosting MVS-based NeRFs to Generalizable View Synthesis in Large-scale Scenes22 Jul 2024
- 2024Differentiable Convex Polyhedra Optimization from Multi-view Images22 Jul 2024
- 2024Fast Learning of Signed Distance Functions from Noisy Point Clouds via Noise to Noise Mapping4 Jul 2024
- 2024PT43D: A Probabilistic Transformer for Generating 3D Shapes from Single Highly-Ambiguous RGB Images20 May 2024
- 2024When LLMs step into the 3D World: A Survey and Meta-Analysis of 3D Tasks via Multi-modal Large Language Models16 May 2024
- 2024Rasterized Edge Gradients: Handling Discontinuities Differentiably3 May 2024
- 2024Probabilistic Directed Distance Fields for Ray-Based Shape Representations13 Apr 2024
- 20243DGSR: Implicit Surface Reconstruction with 3D Gaussian Splatting30 Mar 2024
- 2024PKU-DyMVHumans: A Multi-View Video Benchmark for High-Fidelity Dynamic Human Modeling24 Mar 2024
- 2024UFORecon: Generalizable Sparse-View Surface Reconstruction from Arbitrary and UnFavOrable Sets8 Mar 2024
- 2024AiSDF: Structure-aware Neural Signed Distance Fields in Indoor Scenes4 Mar 2024
- 2024A Concept for Reconstructing Stucco Statues from historic Sketches using synthetic Data only8 Feb 2024
- 2023PR-NeuS: A Prior-based Residual Learning Paradigm for Fast Multi-view Neural Surface Reconstruction18 Dec 2023
- 2023Ins-HOI: Instance Aware Human-Object Interactions Recovery15 Dec 2023
- 2023InceptionHuman: Controllable Prompt-to-NeRF for Photorealistic 3D Human Generation27 Nov 2023
- 2023UP-NeRF: Unconstrained Pose-Prior-Free Neural Radiance Fields7 Nov 2023
- 2023RayDF: Neural Ray-surface Distance Fields with Multi-view Consistency30 Oct 2023
- 2023Ghost on the Shell: An Expressive Representation of General 3D Shapes23 Oct 2023
- 2023Learning Neural Implicit through Volume Rendering with Attentive Depth Fusion Priors17 Oct 2023
- 20234K4D: Real-Time 4D View Synthesis at 4K Resolution17 Oct 2023
- 2023Im4D: High-Fidelity and Real-Time Novel View Synthesis for Dynamic Scenes12 Oct 2023
- 2023OrthoPlanes: A Novel Representation for Better 3D-Awareness of GANs27 Sep 2023
- 2023Automatic Reverse Engineering: Creating computer-aided design (CAD) models from multi-view images23 Sep 2023
- 2023NTO3D: Neural Target Object 3D Reconstruction with Segment Anything22 Sep 2023
- 2023BluNF: Blueprint Neural Field7 Sep 2023
- 2023GridPull: Towards Scalability in Learning Implicit Representations from 3D Point Clouds25 Aug 2023
- 2023Coordinate Quantized Neural Implicit Representations for Multi-view Reconstruction21 Aug 2023
- 2023A Theory of Topological Derivatives for Inverse Rendering of Geometry19 Aug 2023
50 newest of 159 All 159 (JSON)
- 20223D Neural Field Generation using Triplane Diffusioncited by 274
- 2022Dream3D: Zero-Shot Text-to-3D Synthesis Using 3D Shape Prior and Text-to-Image Diffusion Modelscited by 151
- 2023Guide3D: Create 3D Avatars from Text and Image Guidancecited by 10
- 2024Toward General Object-level Mapping from Sparse Views with 3D Diffusion Priorscited by 5
- 2025MV-Performer: Taming Video Diffusion Model for Faithful and Synchronized Multi-view Performer Synthesiscited by 4
- 2025MV-Performer: Taming Video Diffusion Model for Faithful and Synchronized Multi-view Performer Synthesis8 Oct 2025
- 2024Toward General Object-level Mapping from Sparse Views with 3D Diffusion Priors7 Oct 2024
- 2023Guide3D: Create 3D Avatars from Text and Image Guidance18 Aug 2023
- 2022Dream3D: Zero-Shot Text-to-3D Synthesis Using 3D Shape Prior and Text-to-Image Diffusion Models28 Dec 2022
- 20223D Neural Field Generation using Triplane Diffusion30 Nov 2022
- 2021NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstructioncited by 1,712
- 2021Efficient Geometry-aware 3D Generative Adversarial Networkscited by 1,205
- 2021IBRNet: Learning Multi-View Image-Based Renderingcited by 703
- 2021RegNeRF: Regularizing Neural Radiance Fields for View Synthesis from Sparse Inputscited by 615
- 2022MonoSDF: Exploring Monocular Geometric Cues for Neural Implicit Surface Reconstructioncited by 470
- 2023Neuralangelo: High-Fidelity Neural Surface Reconstructioncited by 467
- 2021Neural Geometric Level of Detail: Real-time Rendering with Implicit 3D Shapescited by 426
- 2021Extracting Triangular 3D Models, Materials, and Lighting From Imagescited by 366
- 2021Advances in Neural Renderingcited by 365
- 2021PhySG: Inverse Rendering with Spherical Gaussians for Physics-based Material Editing and Relightingcited by 335
- 2023OmniObject3D: Large-Vocabulary 3D Object Dataset for Realistic Perception, Reconstruction and Generationcited by 326
- 2022NeuS2: Fast Learning of Neural Implicit Surfaces for Multi-view Reconstructioncited by 314
12 most cited of 153 All 153 (JSON)
- 2026Differentiable Voxelization of Surface Representations16 Aug 2026
- 2026Stochastic Signed Distance Processes18 Jun 2026
- 2026S2MDF: A Plug-And-Play Layer for Intersection-Free Multi-Object Signed Distance Fields28 May 2026
- 2025Inverse Rendering for High-Genus Surface Meshes from Multi-View Images24 Nov 2025
- 2025Neural Geometry Image-Based Representations with Optimal Transport (OT)24 Nov 2025
- 2025LAMP: Data-Efficient Linear Affine Weight-Space Models for Parameter-Controlled 3D Shape Generation and Extrapolation26 Oct 2025
- 2025Efficient multi-view training for 3D Gaussian Splatting15 Jun 2025
- 2025Learning to Upscale 3D Segmentations in Neuroimaging27 May 2025
- 2025Sparfels: Fast Reconstruction from Sparse Unposed Imagery4 May 2025
- 2025SparSplat: Fast Multi-View Reconstruction with Generalizable 2D Gaussian Splatting4 May 2025
- 2025Learning Scene-Level Signed Directional Distance Function with Ellipsoidal Priors and Neural Residuals25 Mar 2025
- 2025Glissando-Net: Deep sinGLe vIew category level poSe eStimation ANd 3D recOnstruction24 Jan 2025
12 newest of 153 All 153 (JSON)