mlsanityData to 7 Oct 2026

Relightable 3D Gaussians: Realistic Point Cloud Relighting with BRDF Decomposition and Ray Tracing

arXiv 2311.16043 ↗First posted 27 Nov 2023cs.CV cs.GRcited by 138

Not on the map. Why?

Abstract (arXiv)

In this paper, we present a novel differentiable point-based rendering framework to achieve photo-realistic relighting. To make the reconstructed scene relightable, we enhance vanilla 3D Gaussians by associating extra properties, including normal vectors, BRDF parameters, and incident lighting from various directions. From a collection of multi-view images, the 3D scene is optimized through 3D Gaussian Splatting while BRDF and lighting are decomposed by physically based differentiable rendering. To produce plausible shadow effects in photo-realistic relighting, we introduce an innovative point-based ray tracing with the bounding volume hierarchies for efficient visibility pre-computation. Extensive experiments demonstrate our improved BRDF estimation, novel view synthesis and relighting results compared to state-of-the-art approaches. The proposed framework showcases the potential to revolutionize the mesh-based graphics pipeline with a point-based pipeline enabling editing, tracing, and relighting.

Key facts. Cited by 138 papers in the map (data through 7 Oct 2026). Builds on 43 of its 55 references that are in the map. First posted on arXiv 27 Nov 2023.

Builds on

43 of 55 references.

References · 43

  1. 20233D Gaussian Splatting for Real-Time Radiance Field Renderingcited by 6,141
  2. 2022Instant Neural Graphics Primitives with a Multiresolution Hash Encodingcited by 3,698
  3. 2021Mip-NeRF 360: Unbounded Anti-Aliased Neural Radiance Fieldscited by 2,013
  4. 2021Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fieldscited by 1,957
  5. 2021NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstructioncited by 1,701
  6. 2021Plenoxels: Radiance Fields without Neural Networkscited by 1,607
  7. 2022TensoRF: Tensorial Radiance Fieldscited by 1,275
  8. 2021Volume Rendering of Neural Implicit Surfacescited by 990
  9. 2021Depth-supervised NeRF: Fewer Views and Faster Training for Freecited by 732
  10. 2021KiloNeRF: Speeding up Neural Radiance Fields with Thousands of Tiny MLPscited by 672
  11. 2023Zip-NeRF: Anti-Aliased Grid-Based Neural Radiance Fieldscited by 667
  12. 2023SuGaR: Surface-Aligned Gaussian Splatting for Efficient 3D Mesh Reconstruction and High-Quality Mesh Renderingcited by 611
31 more
  1. 2021Ref-NeRF: Structured View-Dependent Appearance for Neural Radiance Fieldscited by 569
  2. 2021FastNeRF: High-Fidelity Neural Rendering at 200FPScited by 510
  3. 2020NeRV: Neural Reflectance and Visibility Fields for Relighting and View Synthesiscited by 493
  4. 2022MonoSDF: Exploring Monocular Geometric Cues for Neural Implicit Surface Reconstructioncited by 467
  5. 2021Baking Neural Radiance Fields for Real-Time View Synthesiscited by 465
  6. 2020NeRD: Neural Reflectance Decomposition from Image Collectionscited by 446
  7. 2021Extracting Triangular 3D Models, Materials, and Lighting From Imagescited by 363
  8. 2021PhySG: Inverse Rendering with Spherical Gaussians for Physics-based Material Editing and Relightingcited by 332
  9. 2019Neural Point-Based Graphicscited by 326
  10. 2022MobileNeRF: Exploiting the Polygon Rasterization Pipeline for Efficient Neural Field Rendering on Mobile Architecturescited by 308
  11. 2019Differentiable Surface Splatting for Point-based Geometry Processingcited by 238
  12. 2020Neural Reflectance Fields for Appearance Acquisitioncited by 196
  13. 2022Modeling Indirect Illumination for Inverse Renderingcited by 167
  14. 2023TensoIR: Tensorial Inverse Renderingcited by 166
  15. 2022Shape, Light, and Material Decomposition from Images using Monte Carlo Rendering and Denoisingcited by 163
  16. 2023NeRO: Neural Geometry and BRDF Reconstruction of Reflective Objects from Multiview Imagescited by 163
  17. 2022IRON: Inverse Rendering by Optimizing Neural SDFs and Materials from Photometric Imagescited by 110
  18. 2022EfficientNeRF: Efficient Neural Radiance Fieldscited by 110
  19. 2022NeILF: Neural Incident Light Field for Physically-based Material Estimationcited by 105
  20. 2020Deep Reflectance Volumes: Relightable Reconstructions from Multi-View Photometric Imagescited by 98
  21. 2022SAMURAI: Shape And Material from Unconstrained Real-world Arbitrary Image collectionscited by 89
  22. 2022Differentiable Point-Based Radiance Fields for Efficient View Synthesiscited by 86
  23. 2023NeILF++: Inter-Reflectable Light Fields for Geometry and Material Estimationcited by 73
  24. 2019Revealing Scenes by Inverting Structure from Motion Reconstructionscited by 66
  25. 2020Deep 3D Capture: Geometry and Reflectance from Sparse Multi-View Imagescited by 65
  26. 2019Inverse Path Tracing for Joint Material and Lighting Estimationcited by 62
  27. 2023NeFII: Inverse Rendering for Reflectance Decomposition with Near-Field Indirect Illuminationcited by 38
  28. 2023NeMF: Inverse Volume Rendering with Neural Microflake Fieldcited by 34
  29. 2023Flexible Techniques for Differentiable Rendering with 3D Gaussianscited by 28
  30. 2020Seeing the World in a Bag of Chipscited by 25
  31. 2023Frequency-Modulated Point Cloud Rendering with Easy Editingcited by 10

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 22; 2025: 53; 2026: 62.

138 papers.

Sort:

By area:

  1. 2023GIR: 3D Gaussian Inverse Rendering for Relightable Scene Factorizationcited by 86
  2. 20243D Gaussian Splatting: Survey, Technologies, Challenges, and Opportunitiescited by 77
  3. 2024BAGS: Blur Agnostic Gaussian Splatting through Multi-Scale Kernel Modelingcited by 42
  4. 2024Gaussian Splatting: 3D Reconstruction and Novel View Synthesis, a Reviewcited by 40
  5. 2025Matrix3D: Large Photogrammetry Model All-in-Onecited by 39
  6. 2024Ref-GS: Directional Factorization for 2D Gaussian Splattingcited by 27
  7. 2024GauStudio: A Modular Framework for 3D Gaussian Splatting and Beyondcited by 24
  8. 2024ScribbleLight: Single Image Indoor Relighting with Scribblescited by 22
  9. 2025SVG-IR: Spatially-Varying Gaussian Splatting for Inverse Renderingcited by 22
  10. 2025PIN-WM: Learning Physics-INformed World Models for Non-Prehensile Manipulationcited by 22
  11. 2025GUAVA: Generalizable Upper Body 3D Gaussian Avatarcited by 22
  12. 2025MaterialRefGS: Reflective Gaussian Splatting with Multi-view Consistent Material Inferencecited by 22
38 more
  1. 2024Deformable Radial Kernel Splattingcited by 17
  2. 2024Z-Splat: Z-Axis Gaussian Splatting for Camera-Sonar Fusioncited by 16
  3. 2025TransparentGS: Fast Inverse Rendering of Transparent Objects with Gaussianscited by 16
  4. 2025GeoSVR: Taming Sparse Voxels for Geometrically Accurate Surface Reconstructioncited by 15
  5. 2024ReCap: Better Gaussian Relighting with Cross-Environment Capturescited by 14
  6. 2025Unifying Appearance Codes and Bilateral Grids for Driving Scene Gaussian Splattingcited by 14
  7. 2024Leveraging Thermal Modality to Enhance Reconstruction in Low-Light Conditionscited by 12
  8. 2025RainyGS: Efficient Rain Synthesis with Physically-Based Gaussian Splattingcited by 12
  9. 2025Gaussian Splatting with Discretized SDF for Relightable Assetscited by 12
  10. 2024GUS-IR: Gaussian Splatting with Unified Shading for Inverse Renderingcited by 11
  11. 2024GS-ROR2: Bidirectional-guided 3DGS and SDF for Reflective Object Relighting and Reconstructioncited by 10
  12. 2024SweepEvGS: Event-Based 3D Gaussian Splatting for Macro and Micro Radiance Field Rendering from a Single Sweepcited by 10
  13. 2025RTR-GS: 3D Gaussian Splatting for Inverse Rendering with Radiance Transfer and Reflectioncited by 9
  14. 2025TexGS-VolVis: Expressive Scene Editing for Volume Visualization via Textured Gaussian Splattingcited by 9
  15. 2025MV-CoLight: Efficient Object Compositing with Consistent Lighting and Shadow Generationcited by 8
  16. 2025PolGS: Polarimetric Gaussian Splatting for Fast Reflective Surface Reconstructioncited by 8
  17. 2026Generative World Renderercited by 8
  18. 2025GroomLight: Hybrid Inverse Rendering for Relightable Human Hair Appearance Modelingcited by 7
  19. 2025ROGR: Relightable 3D Objects using Generative Relightingcited by 7
  20. 2026LuxRemix: Lighting Decomposition and Remixing for Indoor Scenescited by 7
  21. 2024GS-ProCams: Gaussian Splatting-based Projector-Camera Systemscited by 6
  22. 2025SOF: Sorted Opacity Fields for Fast Unbounded Surface Reconstructioncited by 6
  23. 2026Code as Worlds: Agentic Discovery of Executable World Representations for Physical Reasoningcited by 6
  24. 2024Progressive Radiance Distillation for Inverse Rendering with Gaussian Splattingcited by 5
  25. 2025Hypo3D: Exploring Hypothetical Reasoning in 3Dcited by 5
  26. 2025BEAM: Bridging Physically-based Rendering and Gaussian Modeling for Relightable Volumetric Videocited by 5
  27. 20252D Triangle Splatting for Direct Differentiable Mesh Trainingcited by 5
  28. 2025OLATverse: A Large-scale Real-world Object Dataset with Precise Lighting Controlcited by 5
  29. 2025MVInverse: Feed-forward Multi-view Inverse Rendering in Secondscited by 5
  30. 2026GaussianFluent: Gaussian Simulation for Dynamic Scenes with Mixed Materialscited by 5
  31. 2024PR-ENDO: Physically Based Relightable Gaussian Splatting for Endoscopycited by 4
  32. 2024GeoTexDensifier: Geometry-Texture-Aware Densification for High-Quality Photorealistic 3D Gaussian Splattingcited by 4
  33. 2025From Pixels to Polygons: A Survey of Deep Learning Approaches for Medical Image-to-Mesh Reconstructioncited by 4
  34. 2025Generalizable and Relightable Gaussian Splatting for Human Novel View Synthesiscited by 4
  35. 2025InvRGB+L: Inverse Rendering of Complex Scenes with Unified Color and LiDAR Reflectance Modelingcited by 4
  36. 2025NeAR: Coupled Neural Asset-Renderer Stackcited by 4
  37. 2026Bridging Visual and Wireless Sensing via a Unified Radiation Field for 3D Radio Map Constructioncited by 4
  38. 2026LightCity: An Urban Dataset for Outdoor Inverse Rendering and Reconstruction under Multi-illumination Conditionscited by 4

50 most cited of 138 All 138 (JSON)