mlsanityData to 7 Oct 2026

SViM3D: Stable Video Material Diffusion for Single Image 3D Generation

arXiv 2510.08271 ↗First posted 9 Oct 2025cs.GR cs.CV cs.LGcited by 7

On the map: Diffusion models → 3D, point cloud and scene generation

Abstract (arXiv)

We present Stable Video Materials 3D (SViM3D), a framework to predict multi-view consistent physically based rendering (PBR) materials, given a single image. Recently, video diffusion models have been successfully used to reconstruct 3D objects from a single image efficiently. However, reflectance is still represented by simple material models or needs to be estimated in additional steps to enable relighting and controlled appearance edits. We extend a latent video diffusion model to output spatially varying PBR parameters and surface normals jointly with each generated view based on explicit camera control. This unique setup allows for relighting and generating a 3D asset using our model as neural prior. We introduce various mechanisms to this pipeline that improve quality in this ill-posed setting. We show state-of-the-art relighting and novel view synthesis performance on multiple object-centric datasets. Our method generalizes to diverse inputs, enabling the generation of relightable 3D assets useful in AR/VR, movies, games and other visual media.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Placed in 3D, point cloud and scene generation, one of 14 directions in Diffusion models. Builds on 83 of its 120 references that are in the map. First posted on arXiv 9 Oct 2025.

Builds on

83 of 120 references.

Closest ancestors · 15

Same direction, newest first

  1. 2024MaterialFusion: Enhancing Inverse Rendering with Material Diffusion Priors23 Sep
  2. 2024Neural Gaffer: Relighting Any Object via Diffusion11 Jun
  3. 2024CLAY: A Controllable Large-scale Generative Model for Creating High-quality 3D Assets30 May
  4. 2024RGBleftrightarrowX: Image decomposition and synthesis using material- and lighting-aware diffusion models1 May
  5. 2024SV3D: Novel Multi-view Synthesis and 3D Generation from a Single Image using Latent Video Diffusion18 Mar
  6. 2024V3D: Video Diffusion Models are Effective 3D Generators11 Mar
  7. 2024DiLightNet: Fine-grained Lighting Control for Diffusion-based Image Generation19 Feb
  8. 2023Paint3D: Paint Anything 3D with Lighting-Less Texture Diffusion Models21 Dec
  9. 2023UniDream: Unifying Diffusion Priors for Relightable Text-to-3D Generation14 Dec
  10. 2023RichDreamer: A Generalizable Normal-Depth Diffusion Model for Detail Richness in Text-to-3D28 Nov
  11. 2023ControlMat: A Controlled Generative Approach to Material Capture4 Sep
  12. 2023MATLABER: Material-Aware Text-to-3D via LAtent BRDF auto-EncodeR18 Aug
3 more
  1. 2023Fantasia3D: Disentangling Geometry and Appearance for High-quality Text-to-3D Content Creation24 Mar
  2. 2023Zero-1-to-3: Zero-shot One Image to 3D Object20 Mar
  3. 2022DreamFusion: Text-to-3D using 2D Diffusion29 Sep

Other references · 68

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  4. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
  5. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  6. 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,141
  7. 2022Classifier-Free Diffusion Guidancecited by 5,717
  8. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
  9. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,131
  10. 2022Instant Neural Graphics Primitives with a Multiresolution Hash Encodingcited by 3,698
  11. 2022FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awarenesscited by 3,611
  12. 2022Elucidating the Design Space of Diffusion-Based Generative Modelscited by 2,994
38 more
  1. 2023FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioningcited by 2,551
  2. 2023Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasetscited by 2,526
  3. 2021CLIPScore: A Reference-free Evaluation Metric for Image Captioningcited by 2,388
  4. 2023DUSt3R: Geometric 3D Vision Made Easycited by 1,526
  5. 2020Improved Techniques for Training Score-Based Generative Modelscited by 1,036
  6. 2023LRM: Large Reconstruction Model for Single Image to 3Dcited by 834
  7. 2021RegNeRF: Regularizing Neural Radiance Fields for View Synthesis from Sparse Inputscited by 614
  8. 2022Google Scanned Objects: A High-Quality Dataset of 3D Scanned Household Itemscited by 601
  9. 2021Deep Marching Tetrahedra: a Hybrid Representation for High-Resolution 3D Shape Synthesiscited by 459
  10. 2020NeRD: Neural Reflectance Decomposition from Image Collectionscited by 446
  11. 2021Extracting Triangular 3D Models, Materials, and Lighting From Imagescited by 363
  12. 2021PhySG: Inverse Rendering with Spherical Gaussians for Physics-based Material Editing and Relightingcited by 332
  13. 2022MCVD: Masked Conditional Video Diffusion for Prediction, Generation, and Interpolationcited by 307
  14. 2023Rethinking FID: Towards a Better Evaluation Metric for Image Generationcited by 303
  15. 2021Neural-PIL: Neural Pre-Integrated Lighting for Reflectance Decompositioncited by 204
  16. 2019Inverse Rendering for Complex Indoor Scenes: Shape, Spatially-Varying Lighting and SVBRDF from a Single Imagecited by 202
  17. 2020Neural Reflectance Fields for Appearance Acquisitioncited by 196
  18. 2022Modeling Indirect Illumination for Inverse Renderingcited by 167
  19. 2024MeshLRM: Large Reconstruction Model for High-Quality Meshescited by 138
  20. 2023Relightable Gaussian Codec Avatarscited by 126
  21. 2018Single-Image SVBRDF Capture with a Rendering-Aware Deep Networkcited by 116
  22. 2019Neural Inverse Rendering of an Indoor Scene from a Single Imagecited by 101
  23. 2020Deep Reflectance Volumes: Relightable Reconstructions from Multi-View Photometric Imagescited by 98
  24. 2025MEt3R: Measuring Multi-View Consistency in Generated Imagescited by 91
  25. 2022SAMURAI: Shape And Material from Unconstrained Real-world Arbitrary Image collectionscited by 89
  26. 2023Paint-it: Text-to-Texture Synthesis via Deep Convolutional Texture Map Optimization and Physically-Based Renderingcited by 84
  27. 2022Learning-based Inverse Rendering of Complex Indoor Scenes with Differentiable Monte Carlo Raytracingcited by 78
  28. 2023NeILF++: Inter-Reflectable Light Fields for Geometry and Material Estimationcited by 73
  29. 2023Intrinsic Image Diffusion for Indoor Single-view Material Estimationcited by 73
  30. 2018Materials for Masses: SVBRDF Acquisition with a Single Mobile Phone Imagecited by 66
  31. 2024Meta 3D AssetGen: Text-to-Mesh Generation with High-Quality Geometry, Texture, and PBR Materialscited by 66
  32. 2020Deep 3D Capture: Geometry and Reflectance from Sparse Multi-View Imagescited by 65
  33. 2024SF3D: Stable Fast 3D Mesh Reconstruction with UV-unwrapping and Illumination Disentanglementcited by 65
  34. 2023ENVIDR: Implicit Differentiable Renderer with Neural Environment Lightingcited by 63
  35. 2020Two-shot Spatially-varying BRDF and Shape Estimationcited by 59
  36. 2019Neural Illumination: Lighting Prediction for Indoor Environmentscited by 57
  37. 2024FlashTex: Fast Relightable Mesh Texturing with LightControlNetcited by 56
  38. 2023Stanford-ORB: A Real-World 3D Object Inverse Rendering Benchmarkcited by 51

50 most cited of 68 All 68 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2025: 4; 2026: 3.

7 papers.

  1. 2025NeAR: Coupled Neural Asset-Renderer Stackcited by 4
  2. 2025FrameDiffuser: G-Buffer-Conditioned Diffusion for Neural Forward Frame Renderingcited by 3
  3. 2026VideoMatGen: PBR Materials through Joint Generative Modelingcited by 3
  4. 2025MatSpray: Fusing 2D Material World Knowledge on 3D Geometrycited by 2
  5. 2025FROMAT: Multiview Material Appearance Transfer via Few-Shot Self-Attention Adaptationcited by 0
  6. 2026CAT-Flow: Curvature-Adaptive sTeps for Flow Matchingcited by 0
  7. 2026Texture Space Material Diffusioncited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024MCMat: Multiview-Consistent and Physically Accurate PBR Material Generation
  2. 2026VideoNeuMat: Neural Material Extraction from Generative Video Models
  3. 2024Hi3D: Pursuing High-Resolution Image-to-3D Generation with Video Diffusion Models
  4. 2024Boosting 3D Object Generation through PBR Materials
  5. 2025Yesnt: Are Diffusion Relighting Models Ready for Capture Stage Compositing? A Hybrid Alternative to Bridge the Gap
  6. 2025VideoMat: Extracting PBR Materials from Video Diffusion Models
  7. 2026Diffusion-Based Material Regularization for Physics-Based Inverse Rendering
  8. 2024SV4D: Dynamic 3D Content Generation with Multi-Frame and Multi-View Consistency