# MatSynth: A Modern PBR Materials Dataset

arXiv: https://arxiv.org/abs/2401.06056 · HTML: https://mlsanity.com/paper/2401.06056/

## Key facts

- Cited by 42 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 49 references that are in the map.
- First posted on arXiv 11 Jan 2024.

## Builds on (most cited first, 27 of 27)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022)
- [Analyzing and Improving the Image Quality of StyleGAN](https://mlsanity.com/paper/1912.04958/) (2019)
- [Exploring CLIP for Assessing the Look and Feel of Images](https://mlsanity.com/paper/2207.12396/) (2022)
- [Hypersim: A Photorealistic Synthetic Dataset for Holistic Indoor Scene Understanding](https://mlsanity.com/paper/2011.02523/) (2020)
- [Inverse Rendering for Complex Indoor Scenes: Shape, Spatially-Varying Lighting and SVBRDF from a Single Image](https://mlsanity.com/paper/1905.02722/) (2019)
- [Infinite Photorealistic Worlds using Procedural Generation](https://mlsanity.com/paper/2306.09310/) (2023)
- [Single-Image SVBRDF Capture with a Rendering-Aware Deep Network](https://mlsanity.com/paper/1810.09718/) (2018)
- [Learning-based Inverse Rendering of Complex Indoor Scenes with Differentiable Monte Carlo Raytracing](https://mlsanity.com/paper/2211.03017/) (2022)
- [OpenRooms: An End-to-End Open Framework for Photorealistic Indoor Scene Datasets](https://mlsanity.com/paper/2007.12868/) (2020)
- [IRISformer: Dense Vision Transformers for Single-Image Inverse Rendering in Indoor Scenes](https://mlsanity.com/paper/2206.08423/) (2022)
- [Deep Polarization Imaging for 3D shape and SVBRDF Acquisition](https://mlsanity.com/paper/2105.02875/) (2021)
- [Modeling Surface Appearance from a Single Photograph using Self-augmented Convolutional Neural Networks](https://mlsanity.com/paper/1809.00886/) (2018)
- [Flexible SVBRDF Capture with a Multi-Image Deep Network](https://mlsanity.com/paper/1906.11557/) (2019)
- [ControlMat: A Controlled Generative Approach to Material Capture](https://mlsanity.com/paper/2309.01700/) (2023)
- [MatFuse: Controllable Material Generation with Diffusion Models](https://mlsanity.com/paper/2308.11408/) (2023)
- [Generative Modelling of BRDF Textures from Flash Images](https://mlsanity.com/paper/2102.11861/) (2021)
- [Guided Fine-Tuning for Large-Scale Material Transfer](https://mlsanity.com/paper/2007.03059/) (2020)
- [TileGen: Tileable, Controllable Material Generation and Capture](https://mlsanity.com/paper/2206.05649/) (2022)
- [Materialistic: Selecting Similar Materials in Images](https://mlsanity.com/paper/2305.13291/) (2023)
- [SurfaceNet: Adversarial SVBRDF Estimation from a Single Image](https://mlsanity.com/paper/2107.11298/) (2021)
- [PSDR-Room: Single Photo to Scene using Differentiable Rendering](https://mlsanity.com/paper/2307.03244/) (2023)
- [Generating Procedural Materials from Text or Image Prompts](https://mlsanity.com/paper/2304.13172/) (2023)
- [PhotoMat: A Material Generator Learned from Single Flash Photos](https://mlsanity.com/paper/2305.12296/) (2023)
- [The Visual Language of Fabrics](https://mlsanity.com/paper/2307.13681/) (2023)

## Built on by (most cited first, 30 of 42)

- [Articraft: An Agentic System for Scalable Articulated 3D Asset Generation](https://mlsanity.com/paper/2605.15187/) (2026)
- [Neural LightRig: Unlocking Accurate Object Normal and Material Estimation with Multi-Light Diffusion](https://mlsanity.com/paper/2412.09593/) (2024)
- [LuxDiT: Lighting Estimation with Video Diffusion Transformer](https://mlsanity.com/paper/2509.03680/) (2025)
- [MaterialPicker: Multi-Modal DiT-Based Material Generation](https://mlsanity.com/paper/2412.03225/) (2024)
- [Generative World Renderer](https://mlsanity.com/paper/2604.02329/) (2026)
- [Light of Normals: Unified Feature Representation for Universal Photometric Stereo](https://mlsanity.com/paper/2506.18882/) (2025)
- [MatSwap: Light-aware material transfers in images](https://mlsanity.com/paper/2502.07784/) (2025)
- [SOPHY: Learning to Generate Simulation-Ready Objects with Physical Materials](https://mlsanity.com/paper/2504.12684/) (2025)
- [Chord: Chain of Rendering Decomposition for PBR Material Estimation from Generated Texture Images](https://mlsanity.com/paper/2509.09952/) (2025)
- [OLATverse: A Large-scale Real-world Object Dataset with Precise Lighting Control](https://mlsanity.com/paper/2511.02483/) (2025)
- [StableMaterials: Enhancing Diversity in Material Generation via Semi-Supervised Learning](https://mlsanity.com/paper/2406.09293/) (2024)
- [Vastextures: Vast repository of textures and PBR materials extracted from real-world images using unsupervised methods](https://mlsanity.com/paper/2406.17146/) (2024)
- [Generate Any Scene: Scene Graph Driven Data Synthesis for Visual Generation Training](https://mlsanity.com/paper/2412.08221/) (2024)
- [Fine-Grained Spatially Varying Material Selection in Images](https://mlsanity.com/paper/2506.09023/) (2025)
- [MatPedia: A Universal Generative Foundation for High-Fidelity Material Synthesis](https://mlsanity.com/paper/2511.16957/) (2025)
- [VideoMatGen: PBR Materials through Joint Generative Modeling](https://mlsanity.com/paper/2603.16566/) (2026)
- [Learning Zero-Shot Material States Segmentation, by Implanting Natural Image Patterns in Synthetic Data](https://mlsanity.com/paper/2403.03309/) (2024)
- [MatCLIP: Light- and Shape-Insensitive Assignment of PBR Material Models](https://mlsanity.com/paper/2501.15981/) (2025)
- [Twinner: Shining Light on Digital Twins in a Few Snaps](https://mlsanity.com/paper/2503.08382/) (2025)
- [MatPredict: a dataset and benchmark for learning material properties of diverse indoor objects](https://mlsanity.com/paper/2505.13201/) (2025)
- [Material-informed Gaussian Splatting for 3D World Reconstruction in a Digital Twin](https://mlsanity.com/paper/2511.20348/) (2025)
- [VideoNeuMat: Neural Material Extraction from Generative Video Models](https://mlsanity.com/paper/2602.07272/) (2026)
- [Monocular Normal Estimation via Shading Sequence Estimation](https://mlsanity.com/paper/2602.09929/) (2026)
- [FabricDiffusion: High-Fidelity Texture Transfer for 3D Garments Generation from In-The-Wild Clothing Images](https://mlsanity.com/paper/2410.01801/) (2024)
- [RETRO: REthinking Tactile Representation Learning with Material PriOrs](https://mlsanity.com/paper/2505.14319/) (2025)
- [Generative Perception of Shape and Material from Differential Motion](https://mlsanity.com/paper/2506.02473/) (2025)
- [RealMat: Realistic Materials with Diffusion and Reinforcement Learning](https://mlsanity.com/paper/2509.01134/) (2025)
- [SciTextures: Collecting and Connecting Visual Patterns, Models, and Code Across Science and Art](https://mlsanity.com/paper/2511.01817/) (2025)
- [TexEditor: Structure-Preserving Text-Driven Texture Editing](https://mlsanity.com/paper/2603.18488/) (2026)
- [Relit-LiVE: Relight Video by Jointly Learning Environment Video](https://mlsanity.com/paper/2605.06658/) (2026)

## Abstract (arXiv)

We introduce MatSynth, a dataset of 4,000+ CC0 ultra-high resolution PBR materials. Materials are crucial components of virtual relightable assets, defining the interaction of light at the surface of geometries. Given their importance, significant research effort was dedicated to their representation, creation and acquisition. However, in the past 6 years, most research in material acquisiton or generation relied either on the same unique dataset, or on company-owned huge library of procedural materials. With this dataset we propose a significantly larger, more diverse, and higher resolution set of materials than previously publicly available. We carefully discuss the data collection process and demonstrate the benefits of this dataset on material acquisition and generation applications. The complete data further contains metadata with each material's origin, license, category, tags, creation method and, when available, descriptions and physical size, as well as 3M+ renderings of the augmented materials, in 1K, under various environment lightings. The MatSynth dataset is released through the project page at: https://www.gvecchio.com/matsynth.