mlsanityData to 14 Oct 2026

SuperMat: Physically Consistent PBR Material Estimation at Interactive Rates

arXiv 2411.17515 ↗First posted 26 Nov 2024cs.CVcited by 9

Not on the map. Why?

Abstract (arXiv)

Decomposing physically-based materials from images into their constituent properties remains challenging, particularly when maintaining both computational efficiency and physical consistency. While recent diffusion-based approaches have shown promise, they face substantial computational overhead due to multiple denoising steps and separate models for different material properties. We present SuperMat, a single-step framework that achieves high-quality material decomposition with one-step inference. This enables end-to-end training with perceptual and re-render losses while decomposing albedo, metallic, and roughness maps at millisecond-scale speeds. We further extend our framework to 3D objects through a UV refinement network, enabling consistent material estimation across viewpoints while maintaining efficiency. Experiments demonstrate that SuperMat achieves state-of-the-art PBR material decomposition quality while reducing inference time from seconds to milliseconds per image, and completes PBR material estimation for 3D objects in approximately 3 seconds. The project page is at https://hyj542682306.github.io/SuperMat/.

Key facts. Cited by 9 papers in the map (data through 14 Oct 2026). Builds on 58 of its 71 references that are in the map. First posted on arXiv 26 Nov 2024.

Builds on

58 of 71 references.

References · 58

  1. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  2. 2023Segment Anythingcited by 8,234
  3. 2023DINOv2: Learning Robust Visual Features without Supervisioncited by 7,974
  4. 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,025
  5. 2016Perceptual Losses for Real-Time Style Transfer and Super-Resolutioncited by 2,694
  6. 2022Objaverse: A Universe of Annotated 3D Objectscited by 1,540
  7. 2019Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfercited by 1,495
  8. 2023ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillationcited by 1,219
  9. 2023MVDream: Multi-view Diffusion for 3D Generationcited by 938
  10. 2023Fantasia3D: Disentangling Geometry and Appearance for High-quality Text-to-3D Content Creationcited by 687
  11. 2024CLAY: A Controllable Large-scale Generative Model for Creating High-quality 3D Assetscited by 455
  12. 2021Extracting Triangular 3D Models, Materials, and Lighting From Imagescited by 366
38 more
  1. 2023Common Diffusion Noise Schedules and Sample Steps are Flawedcited by 310
  2. 2023TEXTure: Text-Guided Texturing of 3D Shapescited by 304
  3. 2024GeoWizard: Unleashing the Diffusion Priors for 3D Geometry Estimation from a Single Imagecited by 286
  4. 2022CLIP-Mesh: Generating textured meshes from text using pretrained image-text modelscited by 281
  5. 2023Unleashing Text-to-Image Diffusion Models for Visual Perceptioncited by 269
  6. 2023Text2Tex: Text-driven Texture Synthesis via Diffusion Modelscited by 235
  7. 2023GS-IR: 3D Gaussian Splatting for Inverse Renderingcited by 224
  8. 2021Neural-PIL: Neural Pre-Integrated Lighting for Reflectance Decompositioncited by 205
  9. 2022Modeling Indirect Illumination for Inverse Renderingcited by 168
  10. 2023TensoIR: Tensorial Inverse Renderingcited by 168
  11. 2023RichDreamer: A Generalizable Normal-Depth Diffusion Model for Detail Richness in Text-to-3Dcited by 168
  12. 2024Lotus: Diffusion-based Visual Foundation Model for High-quality Dense Predictioncited by 166
  13. 2022Shape, Light, and Material Decomposition from Images using Monte Carlo Rendering and Denoisingcited by 163
  14. 2024RGBleftrightarrowX: Image decomposition and synthesis using material- and lighting-aware diffusion modelscited by 161
  15. 2024StableNormal: Reducing Diffusion Variance for Stable and Sharp Normalcited by 154
  16. 2023Paint3D: Paint Anything 3D with Lighting-Less Texture Diffusion Modelscited by 123
  17. 2021NeRFReN: Neural Radiance Fields with Reflectionscited by 122
  18. 2018Single-Image SVBRDF Capture with a Rendering-Aware Deep Networkcited by 116
  19. 2022IRON: Inverse Rendering by Optimizing Neural SDFs and Materials from Photometric Imagescited by 111
  20. 2023Monocular Depth Estimation using Diffusion Modelscited by 87
  21. 2023Text-Guided Texturing by Synchronized Multi-View Diffusioncited by 86
  22. 2023Paint-it: Text-to-Texture Synthesis via Deep Convolutional Texture Map Optimization and Physically-Based Renderingcited by 84
  23. 2024Fine-Tuning Image-Conditional Diffusion Models is Easier than You Thinkcited by 83
  24. 2023DiffusionDepth: Diffusion Denoising Approach for Monocular Depth Estimationcited by 82
  25. 2022Learning-based Inverse Rendering of Complex Indoor Scenes with Differentiable Monte Carlo Raytracingcited by 78
  26. 2024DepthFM: Fast Monocular Depth Estimation with Flow Matchingcited by 76
  27. 2023Intrinsic Image Diffusion for Indoor Single-view Material Estimationcited by 73
  28. 2023HyperHuman: Hyper-Realistic Human Generation with Latent Structural Diffusioncited by 68
  29. 2024IntrinsicAnything: Learning Diffusion Priors for Inverse Rendering Under Unknown Illuminationcited by 67
  30. 2024SF3D: Stable Fast 3D Mesh Reconstruction with UV-unwrapping and Illumination Disentanglementcited by 65
  31. 2023Texture Generation on 3D Meshes with Point-UV Diffusioncited by 63
  32. 2024FlashTex: Fast Relightable Mesh Texturing with LightControlNetcited by 56
  33. 2019Unsupervised Learning for Intrinsic Image Decomposition from a Single Imagecited by 55
  34. 2024Meta 3D TextureGen: Fast and Consistent Texture Generation for 3D Objectscited by 52
  35. 2023Stanford-ORB: A Real-World 3D Object Inverse Rendering Benchmarkcited by 51
  36. 2023ControlMat: A Controlled Generative Approach to Material Capturecited by 50
  37. 2024DreamMat: High-quality PBR Material Generation with Geometry- and Light-aware Diffusion Modelscited by 50
  38. 2023UniDream: Unifying Diffusion Priors for Relightable Text-to-3D Generationcited by 47

50 most cited of 58 All 58 (JSON)

Built on by

Cited by year

Citing papers per year: 2025: 9.

9 papers.

  1. 2025MaterialMVP: Illumination-Invariant Material Generation via Multi-view PBR Diffusioncited by 28
  2. 2025MuMA: 3D PBR Texturing via Multi-Channel Multi-View Generation and Agentic Post-Processingcited by 8
  3. 2025Chord: Chain of Rendering Decomposition for PBR Material Estimation from Generated Texture Imagescited by 5
  4. 2025LaFiTe: A Generative Latent Field for 3D Native Texturingcited by 4
  5. 2025Large Material Gaussian Model for Relightable 3D Generationcited by 2
  6. 2025LumiTex: Towards High-Fidelity PBR Texture Generation with Illumination Contextcited by 2
  7. 2025WildCap: Facial Albedo Capture in the Wild via Hybrid Inverse Renderingcited by 1
  8. 2025HiMat: DiT-based Ultra-High Resolution SVBRDF Generationcited by 0
  9. 2025An evaluation of SVBRDF Prediction from Generative Image Models for Appearance Modeling of 3D Scenescited by 0