mlsanityData to 14 Oct 2026

ROSA: Reconstructing Object Shape and Appearance Textures by Adaptive Detail Transfer

arXiv 2501.18595 ↗First posted 30 Jan 2025cs.CVcited by 2

Not on the map. Why?

Abstract (arXiv)

Reconstructing an object's shape and appearance in terms of a mesh textured by a spatially-varying bidirectional reflectance distribution function (SVBRDF) from a limited set of images captured under collocated light is an ill-posed problem. Previous state-of-the-art approaches either aim to reconstruct the appearance directly on the geometry or additionally use texture normals as part of the appearance features. However, this requires detailed but inefficiently large meshes, that would have to be simplified in a post-processing step, or suffers from well-known limitations of normal maps such as missing shadows or incorrect silhouettes. Another limiting factor is the fixed and typically low resolution of the texture estimation resulting in loss of important surface details. To overcome these problems, we present ROSA, an inverse rendering method that directly optimizes mesh geometry with spatially adaptive mesh resolution solely based on the image data. In particular, we refine the mesh and locally condition the surface smoothness based on the estimated normal texture and mesh curvature. In addition, we enable the reconstruction of fine appearance details in high-resolution textures through a pioneering tile-based method that operates on a single pre-trained decoder network but is not limited by the network output resolution.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 37 of its 61 references that are in the map. First posted on arXiv 30 Jan 2025.

Builds on

37 of 61 references.

References · 37

  1. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,177
  2. 2021NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstructioncited by 1,712
  3. 2021Volume Rendering of Neural Implicit Surfacescited by 996
  4. 2019Scene Representation Networks: Continuous 3D-Structure-Aware Neural Scene Representationscited by 864
  5. 2019Differentiable Volumetric Rendering: Learning Implicit 3D Representations without 3D Supervisioncited by 700
  6. 2021UNISURF: Unifying Neural Implicit Surfaces and Radiance Fields for Multi-View Reconstructioncited by 646
  7. 2020NeRV: Neural Reflectance and Visibility Fields for Relighting and View Synthesiscited by 496
  8. 2020NeRD: Neural Reflectance Decomposition from Image Collectionscited by 449
  9. 2020Modular Primitives for High-Performance Differentiable Renderingcited by 441
  10. 2021Extracting Triangular 3D Models, Materials, and Lighting From Imagescited by 366
  11. 2021PhySG: Inverse Rendering with Spherical Gaussians for Physics-based Material Editing and Relightingcited by 335
  12. 2022MobileNeRF: Exploiting the Polygon Rasterization Pipeline for Efficient Neural Field Rendering on Mobile Architecturescited by 311
25 more
  1. 2019Deferred Neural Rendering: Image Synthesis using Neural Texturescited by 253
  2. 2020Neural Reflectance Fields for Appearance Acquisitioncited by 196
  3. 2019Neural Volumes: Learning Dynamic Renderable Volumes from Imagescited by 173
  4. 2023TensoIR: Tensorial Inverse Renderingcited by 168
  5. 2022Shape, Light, and Material Decomposition from Images using Monte Carlo Rendering and Denoisingcited by 163
  6. 2022NeuMesh: Learning Disentangled Neural Mesh-based Implicit Field for Geometry and Texture Editingcited by 142
  7. 2021NeRFactor: Neural Factorization of Shape and Reflectance Under an Unknown Illuminationcited by 135
  8. 2022IRON: Inverse Rendering by Optimizing Neural SDFs and Materials from Photometric Imagescited by 111
  9. 2021NeuTex: Neural Texture Mapping for Volumetric Neural Renderingcited by 98
  10. 2022SAMURAI: Shape And Material from Unconstrained Real-world Arbitrary Image collectionscited by 89
  11. 2021Unified Shape and SVBRDF Recovery using Differentiable Monte Carlo Renderingcited by 71
  12. 2022PS-NeRF: Neural Inverse Rendering for Multi-view Photometric Stereocited by 69
  13. 2023Ref-NeuS: Ambiguity-Reduced Neural Implicit Surface Learning for Multi-View Reconstruction with Reflectioncited by 69
  14. 2020Deep 3D Capture: Geometry and Reflectance from Sparse Multi-View Imagescited by 65
  15. 2023ENVIDR: Implicit Differentiable Renderer with Neural Environment Lightingcited by 65
  16. 2020Two-shot Spatially-varying BRDF and Shape Estimationcited by 59
  17. 2023Neural-PBIR Reconstruction of Shape, Material, and Illuminationcited by 45
  18. 2023NeFII: Inverse Rendering for Reflectance Decomposition with Near-Field Indirect Illuminationcited by 38
  19. 2023Neural Microfacet Fields for Inverse Renderingcited by 38
  20. 2023NeMF: Inverse Volume Rendering with Neural Microflake Fieldcited by 34
  21. 2020MaterialGAN: Reflectance Capture using a Generative SVBRDF Modelcited by 33
  22. 2023Factored-NeuS: Reconstructing Surfaces, Illumination, and Materials of Possibly Glossy Objectscited by 15
  23. 2020Shape From Tracing: Towards Reconstructing 3D Object Geometry and SVBRDF Material from Images via Differentiable Path Tracingcited by 10
  24. 2021Revisiting Point Cloud Simplification: A Learnable Feature Preserving Approachcited by 9
  25. 2022SupeRVol: Super-Resolution Shape and Reflectance Estimation in Inverse Volume Renderingcited by 6

Built on by

Cited by year

Citing papers per year: 2025: 2.

2 papers.

  1. 2025SAFT: Shape and Appearance of Fabrics from Template via Differentiable Physical Simulations from Monocular Videocited by 0
  2. 2025Improving Multi-View Reconstruction via Texture-Guided Gaussian-Mesh Joint Optimizationcited by 0