Reflecting Process Expertise in Procedural Material Generation
Not on the map. Why?
Abstract (arXiv)
Procedural material creation underpins applications in digital content creation, visual effects, and 3D asset design. Achieving high-quality results requires more than reproducing node graphs -- it demands understanding the process by which experts construct materials. We formulate procedural material generation as retrieval-time process reasoning over expert demonstrations, elevating process to a first-class representation beyond graph-only synthesis. Concretely, we represent expert workflows as process traces: textual records of construction steps, parameters, and design intent. To instantiate this idea, we use a pretrained LLM-based ProcessSynthesizer to synthesize a process trace aligned with a user's intent and a pretrained LLM-based Compiler to ground the process trace into an executable Blender material graph. Because procedural expertise is most naturally conveyed through demonstrations, we leverage tutorial videos as a source of process knowledge and extract textual, LLM-compatible traces using automated video analysis tools. In an expert study with five Blender artists (avg. 7.5 years of experience), materials generated by reflecting expert demonstrations were found to produce workflows requiring fewer edits, and more closely match professional design strategies than methods operating solely on static artifacts. A user study with 150 participants further shows that our approach achieves superior generation and editing performance compared to prior procedural systems. All code, models, and data will be available at https://materialapprentice.github.io
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 20 of its 27 references that are in the map. First posted on arXiv 14 Jul 2026.
Builds on
20 of 27 references.
References · 20
- 2022ReAct: Synergizing Reasoning and Acting in Language Modelscited by 8,355
- 2021Evaluating Large Language Models Trained on Codecited by 8,053
- 2019Analyzing and Improving the Image Quality of StyleGANcited by 3,580
- 2023Self-Refine: Iterative Refinement with Self-Feedbackcited by 3,317
- 2021Masked-attention Mask Transformer for Universal Image Segmentationcited by 2,338
- 2023LayoutGPT: Compositional Visual Planning and Generation with Large Language Modelscited by 337
- 2023Holodeck: Language Guided Generation of 3D Embodied AI Environmentscited by 257
- 2024SceneCraft: An LLM Agent for Synthesizing 3D Scene as Blender Codecited by 125
- 2025CAST: Component-Aligned 3D Scene Reconstruction from an RGB Imagecited by 101
- 2024LayoutVLM: Differentiable Optimization of 3D Layout via Vision-Language Modelscited by 94
- 20233D-GPT: Procedural 3D Modeling with Large Language Modelscited by 91
- 2023ControlMat: A Controlled Generative Approach to Material Capturecited by 50
8 more
- 2024DreamMat: High-quality PBR Material Generation with Geometry- and Light-aware Diffusion Modelscited by 50
- 2024Material Anything: Generating Materials for Any 3D Object via Diffusioncited by 31
- 2024L3GO: Language Agents with Chain-of-3D-Thoughts for Generating Unconventional Objectscited by 26
- 2023Generating Procedural Materials from Text or Image Promptscited by 21
- 2024BlenderAlchemy: Editing 3D Graphics with Vision-Language Modelscited by 18
- 2024MaterialPicker: Multi-Modal DiT-Based Material Generationcited by 10
- 2025VLMaterial: Procedural Material Generation with Large Vision-Language Modelscited by 9
- 2024An evaluation of LLM code generation capabilities through graded exercisescited by 3
Built on by
Citing papers per year: 2026: 1.
1 paper.