mlsanityData to 7 Oct 2026

Inverse Rendering of Translucent Objects using Physical and Neural Renderers

arXiv 2305.08336 ↗First posted 15 May 2023cs.CVcited by 9

Not on the map. Why?

Abstract (arXiv)

In this work, we propose an inverse rendering model that estimates 3D shape, spatially-varying reflectance, homogeneous subsurface scattering parameters, and an environment illumination jointly from only a pair of captured images of a translucent object. In order to solve the ambiguity problem of inverse rendering, we use a physically-based renderer and a neural renderer for scene reconstruction and material editing. Because two renderers are differentiable, we can compute a reconstruction loss to assist parameter estimation. To enhance the supervision of the proposed neural renderer, we also propose an augmented loss. In addition, we use a flash and no-flash image pair as the input. To supervise the training, we constructed a large-scale synthetic dataset of translucent objects, which consists of 117K scenes. Qualitative and quantitative results on both synthetic and real-world datasets demonstrated the effectiveness of the proposed model.

Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Builds on 43 of its 75 references that are in the map. First posted on arXiv 15 May 2023.

Builds on

43 of 75 references.

References · 43

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2016Image-to-Image Translation with Conditional Adversarial Networkscited by 4,510
  3. 2015ShapeNet: An Information-Rich 3D Model Repositorycited by 2,880
  4. 2019Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfercited by 1,484
  5. 2018Deep Ordinal Regression Network for Monocular Depth Estimationcited by 721
  6. 2017Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networkscited by 707
  7. 2020NeRD: Neural Reflectance Decomposition from Image Collectionscited by 446
  8. 2021PhySG: Inverse Rendering with Spherical Gaussians for Physics-based Material Editing and Relightingcited by 332
  9. 2021Neural-PIL: Neural Pre-Integrated Lighting for Reflectance Decompositioncited by 204
  10. 2019Inverse Rendering for Complex Indoor Scenes: Shape, Spatially-Varying Lighting and SVBRDF from a Single Imagecited by 202
  11. 2017Learning to Predict Indoor Illumination from a Single Imagecited by 160
  12. 2015Deep Convolutional Inverse Graphics Networkcited by 121
31 more
  1. 2019Neural Rerendering in the Wildcited by 121
  2. 2018Single-Image SVBRDF Capture with a Rendering-Aware Deep Networkcited by 116
  3. 2017SfSNet: Learning Shape, Reflectance and Illuminance of Faces in the Wildcited by 110
  4. 2019Neural Inverse Rendering of an Indoor Scene from a Single Imagecited by 101
  5. 2020Image GANs meet Differentiable Rendering for Inverse Graphics and Interpretable 3D Neural Renderingcited by 101
  6. 2018InverseRenderNet: Learning single image inverse renderingcited by 100
  7. 2021Learning Indoor Inverse Rendering with 3D Spatially-Varying Lightingcited by 82
  8. 2018Materials for Masses: SVBRDF Acquisition with a Single Mobile Phone Imagecited by 66
  9. 2020Deep 3D Capture: Geometry and Reflectance from Sparse Multi-View Imagescited by 65
  10. 2017CycleGAN, a Master of Steganographycited by 63
  11. 2022IRISformer: Dense Vision Transformers for Single-Image Inverse Rendering in Indoor Scenescited by 62
  12. 2020Two-shot Spatially-varying BRDF and Shape Estimationcited by 59
  13. 2021Deep Polarization Imaging for 3D shape and SVBRDF Acquisitioncited by 56
  14. 2016Deep Outdoor Illumination Estimationcited by 53
  15. 2021DIB-R++: Learning to Predict Lighting and Material with a Hybrid Differentiable Renderercited by 52
  16. 2018Modeling Surface Appearance from a Single Photograph using Self-augmented Convolutional Neural Networkscited by 51
  17. 2021Self-supervised Outdoor Scene Relightingcited by 48
  18. 2016Deep Shading: Convolutional Neural Networks for Screen-Space Shadingcited by 46
  19. 2016Learning Non-Lambertian Object Intrinsics across ShapeNet Categoriescited by 42
  20. 2018Deep attention-based classification network for robust depth predictioncited by 36
  21. 2020Learning to Factorize and Relight a Citycited by 36
  22. 2021Shape and Material Capture at Homecited by 36
  23. 2018LIME: Live Intrinsic Material Estimationcited by 33
  24. 2020Through the Looking Glass: Neural 3D Reconstruction of Transparent Shapescited by 32
  25. 2022De-rendering 3D Objects in the Wildcited by 31
  26. 2021De-rendering the World's Revolutionary Artefactscited by 28
  27. 20183D-Aware Scene Manipulation via Inverse Graphicscited by 24
  28. 2017Material Editing Using a Physically Based Rendering Networkcited by 17
  29. 2021Self-Supervised 3D Face Reconstruction via Conditional Estimationcited by 17
  30. 2016What Is Around The Camera?cited by 10
  31. 2019Single Image BRDF Parameter Estimation with a Conditional Adversarial Networkcited by 6

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 2; 2024: 1; 2025: 3; 2026: 3.

9 papers.

  1. 2023NeISF: Neural Incident Stokes Field for Geometry and Material Estimationcited by 25
  2. 2023Diffusion Reflectance Map: Single-Image Stochastic Inverse Rendering of Illumination and Reflectancecited by 11
  3. 2025DNF-Intrinsic: Deterministic Noise-Free Diffusion for Indoor Inverse Renderingcited by 11
  4. 2024Deep Polarization Cues for Single-shot Shape and Subsurface Scattering Estimationcited by 6
  5. 2025NPSim: Nighttime Photorealistic Simulation From Daytime Images With Monocular Inverse Rendering and Ray Tracingcited by 0
  6. 2025TensoIS: A Step Towards Feed-Forward Tensorial Inverse Subsurface Scattering for Perlin Distributed Heterogeneous Mediacited by 0
  7. 2026DIAMOND-SSS: Diffusion-Augmented Multi-View Optimization for Data-efficient SubSurface Scatteringcited by 0
  8. 2026GTSR: Subsurface Scattering Awared 3D Gaussians for Translucent Surface Reconstructioncited by 0
  9. 2026FlashNormal: Detailed Surface Normal Estimation from Flash and No-Flash Imagescited by 0