mlsanityData to 7 Oct 2026

IRON: Inverse Rendering by Optimizing Neural SDFs and Materials from Photometric Images

arXiv 2204.02232 ↗First posted 5 Apr 2022cs.CVcited by 110

Not on the map. Why?

Abstract (arXiv)

We propose a neural inverse rendering pipeline called IRON that operates on photometric images and outputs high-quality 3D content in the format of triangle meshes and material textures readily deployable in existing graphics pipelines. Our method adopts neural representations for geometry as signed distance fields (SDFs) and materials during optimization to enjoy their flexibility and compactness, and features a hybrid optimization scheme for neural SDFs: first, optimize using a volumetric radiance field approach to recover correct topology, then optimize further using edgeaware physics-based surface rendering for geometry refinement and disentanglement of materials and lighting. In the second stage, we also draw inspiration from mesh-based differentiable rendering, and design a novel edge sampling algorithm for neural SDFs to further improve performance. We show that our IRON achieves significantly better inverse rendering quality compared to prior works. Our project page is here: https://kai-46.github.io/IRON-website/

Key facts. Cited by 110 papers in the map (data through 7 Oct 2026). Builds on 18 of its 43 references that are in the map. First posted on arXiv 5 Apr 2022.

Builds on

18 of 43 references.

References · 18

  1. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  2. 2019DeepSDF: Learning Continuous Signed Distance Functions for Shape Representationcited by 2,682
  3. 2016Perceptual Losses for Real-Time Style Transfer and Super-Resolutioncited by 2,678
  4. 2020Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domainscited by 2,170
  5. 2018Occupancy Networks: Learning 3D Reconstruction in Function Spacecited by 2,037
  6. 2021NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstructioncited by 1,701
  7. 2021Volume Rendering of Neural Implicit Surfacescited by 990
  8. 2020Implicit Geometric Regularization for Learning Shapescited by 707
  9. 2019Differentiable Volumetric Rendering: Learning Implicit 3D Representations without 3D Supervisioncited by 699
  10. 2021UNISURF: Unifying Neural Implicit Surfaces and Radiance Fields for Multi-View Reconstructioncited by 643
  11. 2020NeRD: Neural Reflectance Decomposition from Image Collectionscited by 446
  12. 2020Modular Primitives for High-Performance Differentiable Renderingcited by 439
6 more
  1. 2021PhySG: Inverse Rendering with Spherical Gaussians for Physics-based Material Editing and Relightingcited by 332
  2. 2019Texture Fields: Learning Texture Representations in Function Spacecited by 210
  3. 2020Deep Reflectance Volumes: Relightable Reconstructions from Multi-View Photometric Imagescited by 98
  4. 2021Unified Shape and SVBRDF Recovery using Differentiable Monte Carlo Renderingcited by 71
  5. 2020Deep 3D Capture: Geometry and Reflectance from Sparse Multi-View Imagescited by 65
  6. 2021Differentiable Surface Rendering via Non-Differentiable Samplingcited by 28

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 9; 2023: 43; 2024: 26; 2025: 22; 2026: 10.

110 papers.

Sort:

By area:

  1. 2023GS-IR: 3D Gaussian Splatting for Inverse Renderingcited by 220
  2. 2023TensoIR: Tensorial Inverse Renderingcited by 166
  3. 2023NeRO: Neural Geometry and BRDF Reconstruction of Reflective Objects from Multiview Imagescited by 163
  4. 2023Relightable 3D Gaussians: Realistic Point Cloud Relighting with BRDF Decomposition and Ray Tracingcited by 138
  5. 2024Neural Gaffer: Relighting Any Object via Diffusioncited by 91
  6. 2023Neural Fields meet Explicit Geometric Representation for Inverse Rendering of Urban Scenescited by 89
  7. 2022Object-Compositional Neural Implicit Surfacescited by 74
  8. 2024IntrinsicAnything: Learning Diffusion Priors for Inverse Rendering Under Unknown Illuminationcited by 67
  9. 2024Advances in 3D Generation: A Surveycited by 62
  10. 2022Interactive Segmentation of Radiance Fieldscited by 60
  11. 2025DiffusionRenderer: Neural Inverse and Forward Rendering with Video Diffusion Modelscited by 55
  12. 2023Stanford-ORB: A Real-World 3D Object Inverse Rendering Benchmarkcited by 51
38 more
  1. 2022NeuPhysics: Editable Neural Geometry and Physics from Monocular Videoscited by 50
  2. 2024DreamMat: High-quality PBR Material Generation with Geometry- and Light-aware Diffusion Modelscited by 50
  3. 2023Relighting Neural Radiance Fields with Shadow and Highlight Hintscited by 41
  4. 2023ObjectSDF++: Improved Object-Compositional Neural Implicit Surfacescited by 38
  5. 2025UniRelight: Learning Joint Decomposition and Synthesis for Video Relightingcited by 38
  6. 2023Neural Microfacet Fields for Inverse Renderingcited by 37
  7. 2023NeMF: Inverse Volume Rendering with Neural Microflake Fieldcited by 34
  8. 2023DreamSparse: Escaping from Plato's Cave with 2D Frozen Diffusion Model Given Sparse Viewscited by 31
  9. 2023OpenIllumination: A Multi-Illumination Dataset for Inverse Rendering Evaluation on Real Objectscited by 31
  10. 2022S3-NeRF: Neural Reflectance Field from Shading and Shadow under a Single Viewpointcited by 30
  11. 2023Scalable, Detailed and Mask-Free Universal Photometric Stereocited by 30
  12. 2025Digital Twin Catalog: A Large-Scale Photorealistic 3D Object Digital Twin Datasetcited by 29
  13. 2024MaterialFusion: Enhancing Inverse Rendering with Material Diffusion Priorscited by 28
  14. 2023NeuManifold: Neural Watertight Manifold Reconstruction with Efficient and High-Quality Rendering Supportcited by 25
  15. 2023NeISF: Neural Incident Stokes Field for Geometry and Material Estimationcited by 25
  16. 2022Multi-view Inverse Rendering for Large-scale Real-world Indoor Scenescited by 23
  17. 2024RelitLRM: Generative Relightable Radiance for Large Reconstruction Modelscited by 23
  18. 2023UrbanIR: Large-Scale Urban Scene Inverse Rendering from a Single Videocited by 22
  19. 2025SVG-IR: Spatially-Varying Gaussian Splatting for Inverse Renderingcited by 22
  20. 2022High-Res Facial Appearance Capture from Polarized Smartphone Imagescited by 21
  21. 2023Dynamic Mesh-Aware Radiance Fieldscited by 21
  22. 2024IDArb: Intrinsic Decomposition for Arbitrary Number of Input Views and Illuminationscited by 21
  23. 2023WildLight: In-the-wild Inverse Rendering with a Flashlightcited by 20
  24. 2023SIRe-IR: Inverse Rendering for BRDF Reconstruction with Shadow and Illumination Removal in High-Illuminance Scenescited by 19
  25. 2022ShadowNeuS: Neural SDF Reconstruction by Shadow Ray Supervisioncited by 17
  26. 2025LIRM: Large Inverse Rendering Model for Progressive Reconstruction of Shape, Materials and View-dependent Radiance Fieldscited by 16
  27. 2023Learning Object-Centric Neural Scattering Functions for Free-Viewpoint Relighting and Scene Compositioncited by 15
  28. 2023VDN-NeRF: Resolving Shape-Radiance Ambiguity via View-Dependence Normalizationcited by 15
  29. 2023Factored-NeuS: Reconstructing Surfaces, Illumination, and Materials of Possibly Glossy Objectscited by 15
  30. 2023Accidental Light Probescited by 13
  31. 2024RNG: Relightable Neural Gaussianscited by 13
  32. 2024Surfel-based Gaussian Inverse Rendering for Fast and Relightable Dynamic Human Reconstruction from Monocular Videocited by 12
  33. 2025Gaussian Splatting with Discretized SDF for Relightable Assetscited by 12
  34. 2023Relightable Neural Actor with Intrinsic Decomposition and Pose Controlcited by 11
  35. 2024GUS-IR: Gaussian Splatting with Unified Shading for Inverse Renderingcited by 11
  36. 2024GS-ROR2: Bidirectional-guided 3DGS and SDF for Reflective Object Relighting and Reconstructioncited by 10
  37. 2025Materialist: Physically Based Editing Using Single-Image Inverse Renderingcited by 10
  38. 2023High-Quality Facial Geometry and Appearance Capture at Homecited by 9

50 most cited of 110 All 110 (JSON)