mlsanityData to 7 Oct 2026

ReLight My NeRF: A Dataset for Novel View Synthesis and Relighting of Real World Objects

arXiv 2304.10448 ↗First posted 20 Apr 2023cs.CVcited by 42

Not on the map. Why?

Abstract (arXiv)

In this paper, we focus on the problem of rendering novel views from a Neural Radiance Field (NeRF) under unobserved light conditions. To this end, we introduce a novel dataset, dubbed ReNe (Relighting NeRF), framing real world objects under one-light-at-time (OLAT) conditions, annotated with accurate ground-truth camera and light poses. Our acquisition pipeline leverages two robotic arms holding, respectively, a camera and an omni-directional point-wise light source. We release a total of 20 scenes depicting a variety of objects with complex geometry and challenging materials. Each scene includes 2000 images, acquired from 50 different points of views under 40 different OLAT conditions. By leveraging the dataset, we perform an ablation study on the relighting capability of variants of the vanilla NeRF architecture and identify a lightweight architecture that can render novel views of an object under novel light conditions, which we use to establish a non-trivial baseline for the dataset. Dataset and benchmark are available at https://eyecan-ai.github.io/rene.

Key facts. Cited by 42 papers in the map (data through 7 Oct 2026). Builds on 47 of its 79 references that are in the map. First posted on arXiv 20 Apr 2023.

Builds on

47 of 79 references.

References · 47

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2022Instant Neural Graphics Primitives with a Multiresolution Hash Encodingcited by 3,698
  3. 2021Mip-NeRF: A Multiscale Representation for Anti-Aliasing Neural Radiance Fieldscited by 1,957
  4. 2021NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstructioncited by 1,701
  5. 2020pixelNeRF: Neural Radiance Fields from One or Few Imagescited by 1,574
  6. 2020D-NeRF: Neural Radiance Fields for Dynamic Scenescited by 1,459
  7. 2020NeRF in the Wild: Neural Radiance Fields for Unconstrained Photo Collectionscited by 1,230
  8. 2020Nerfies: Deformable Neural Radiance Fieldscited by 1,194
  9. 2020Neural Sparse Voxel Fieldscited by 959
  10. 2021PlenOctrees for Real-time Rendering of Neural Radiance Fieldscited by 904
  11. 2020Neural Scene Flow Fields for Space-Time View Synthesis of Dynamic Scenescited by 750
  12. 2020pi-GAN: Periodic Implicit Generative Adversarial Networks for 3D-Aware Image Synthesiscited by 708
35 more
  1. 2020GRAF: Generative Radiance Fields for 3D-Aware Image Synthesiscited by 688
  2. 2019Local Light Field Fusion: Practical View Synthesis with Prescriptive Sampling Guidelinescited by 672
  3. 2021KiloNeRF: Speeding up Neural Radiance Fields with Thousands of Tiny MLPscited by 672
  4. 2020NeRV: Neural Reflectance and Visibility Fields for Relighting and View Synthesiscited by 493
  5. 2020Non-Rigid Neural Radiance Fields: Reconstruction and Novel View Synthesis of a Dynamic Scene From Monocular Videocited by 470
  6. 2020Dynamic Neural Radiance Fields for Monocular 4D Facial Avatar Reconstructioncited by 465
  7. 2020NeRD: Neural Reflectance Decomposition from Image Collectionscited by 446
  8. 2021Neural Geometric Level of Detail: Real-time Rendering with Implicit 3D Shapescited by 426
  9. 2021Dynamic View Synthesis from Dynamic Monocular Videocited by 420
  10. 2020Space-time Neural Irradiance Fields for Free-Viewpoint Videocited by 414
  11. 2018DeepVoxels: Learning Persistent 3D Feature Embeddingscited by 401
  12. 2020Neural Scene Graphs for Dynamic Scenescited by 298
  13. 2021Mixture of Volumetric Primitives for Efficient Neural Renderingcited by 277
  14. 2020Single-View View Synthesis with Multiplane Imagescited by 263
  15. 2020Learned Initializations for Optimizing Coordinate-Based Neural Representationscited by 240
  16. 2020AutoInt: Automatic Integration for Fast Neural Volume Renderingcited by 204
  17. 2020Neural Reflectance Fields for Appearance Acquisitioncited by 196
  18. 2021Neural Articulated Radiance Fieldcited by 196
  19. 2019Pushing the Boundaries of View Extrapolation with Multiplane Imagescited by 191
  20. 2020GRF: Learning a General Radiance Field for 3D Representation and Renderingcited by 183
  21. 2021CodeNeRF: Disentangled Neural Radiance Fields for Object Categoriescited by 183
  22. 2020DeRF: Decomposed Radiance Fieldscited by 163
  23. 2019Single Image Portrait Relightingcited by 152
  24. 2021NeRFactor: Neural Factorization of Shape and Reflectance Under an Unknown Illuminationcited by 135
  25. 2021NeRF-VAE: A Geometry Aware 3D Scene Generative Modelcited by 111
  26. 2021Neural Actor: Neural Free-view Synthesis of Human Actors with Pose Controlcited by 106
  27. 2020Deep Reflectance Volumes: Relightable Reconstructions from Multi-View Photometric Imagescited by 98
  28. 2021ShaRF: Shape-conditioned Radiance Fields from a Single Viewcited by 92
  29. 2019Learning Physics-guided Face Relighting under Directional Lightcited by 90
  30. 2020Neural Light Transport for Relighting and View Synthesiscited by 73
  31. 2020Crowdsampling the Plenoptic Functioncited by 62
  32. 2021CAMPARI: Camera-Aware Decomposed Generative Neural Radiance Fieldscited by 55
  33. 2022Neural Radiance Transfer Fields for Relightable Novel-view Synthesis with Global Illuminationcited by 41
  34. 2021NeLF: Neural Light-transport Field for Portrait View Synthesis and Relightingcited by 38
  35. 2020Light Stage Super-Resolution: Continuous High-Frequency Relightingcited by 20

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 7; 2024: 17; 2025: 11; 2026: 7.

42 papers.

Sort:

By area:

  1. 2023Stanford-ORB: A Real-World 3D Object Inverse Rendering Benchmarkcited by 51
  2. 2024A Diffusion Approach to Radiance Field Relighting using Multi-Illumination Synthesiscited by 37
  3. 2024LumiNet: Latent Intrinsics Meets Diffusion Models for Indoor Scene Relightingcited by 37
  4. 2023OpenIllumination: A Multi-Illumination Dataset for Inverse Rendering Evaluation on Real Objectscited by 31
  5. 2025Digital Twin Catalog: A Large-Scale Photorealistic 3D Object Digital Twin Datasetcited by 29
  6. 2023SuperNormal: Neural Surface Reconstruction via Multi-View Normal Integrationcited by 18
  7. 2024Objects With Lighting: A Real-World Dataset for Evaluating Reconstruction and Rendering for Object Relightingcited by 18
  8. 2025Personalized Vision via Visual In-Context Learningcited by 14
  9. 2025Comprehensive Relighting: Generalizable and Consistent Monocular Human Relighting and Harmonizationcited by 12
  10. 2025GaRe: Relightable 3D Gaussian Splatting for Outdoor Scenes from Unconstrained Photo Collectionscited by 11
  11. 2023Robo360: A 3D Omnispective Multi-Material Robotic Manipulation Datasetcited by 8
  12. 2024BiGS: Bidirectional Gaussian Primitives for Relightable 3D Gaussian Splattingcited by 8
30 more
  1. 2023Fast Sun-aligned Outdoor Scene Relighting based on TensoRFcited by 7
  2. 2023Lasagna: Layered Score Distillation for Disentangled Object Relightingcited by 7
  3. 2025HumanOLAT: A Large-Scale Dataset for Full-Body Human Relighting and Novel-View Synthesiscited by 7
  4. 2023Artist-Friendly Relightable and Animatable Neural Headscited by 6
  5. 2024SMURF: Continuous Dynamics for Motion-Deblurring Radiance Fieldscited by 6
  6. 2024Relighting from a Single Image: Datasets and Deep Intrinsic-based Architecturecited by 6
  7. 2024Relighting Scenes with Object Insertions in Neural Radiance Fieldscited by 5
  8. 2025OLATverse: A Large-scale Real-world Object Dataset with Precise Lighting Controlcited by 5
  9. 2024MetaGS: A Meta-Learned Gaussian-Phong Model for Out-of-Distribution 3D Scene Relightingcited by 4
  10. 2025After the Party: Navigating the Mapping From Color to Ambient Lightingcited by 4
  11. 2025A Real-world Display Inverse Rendering Datasetcited by 4
  12. 2026LightCity: An Urban Dataset for Outdoor Inverse Rendering and Reconstruction under Multi-illumination Conditionscited by 4
  13. 2024RRM: Relightable assets using Radiance guided Material extractioncited by 3
  14. 2025Robust Low-light Scene Restoration via Illumination Transitioncited by 3
  15. 2024ESR-NeRF: Emissive Source Reconstruction Using LDR Multi-view Imagescited by 2
  16. 2024PIR: Photometric Inverse Rendering with Shading Cues Modeling and Surface Reflectance Regularizationcited by 2
  17. 2024Learning autonomous driving from aerial imagerycited by 2
  18. 2025Can NeRFs See without Cameras?cited by 2
  19. 2024Material Transforms from Disentangled NeRF Representationscited by 1
  20. 2024MLI-NeRF: Multi-Light Intrinsic-Aware Neural Radiance Fieldscited by 1
  21. 2025POLAR: A Portrait OLAT Dataset and Generative Framework for Illumination-Aware Face Modelingcited by 1
  22. 2026Learning Latent Proxies for Controllable Single-Image Relightingcited by 1
  23. 2024OpenMaterial: A Large-scale Dataset of Complex Materials for 3D Reconstructioncited by 0
  24. 2024Learning Relighting and Intrinsic Decomposition in Neural Radiance Fieldscited by 0
  25. 2024Incorporating dense metric depth into neural 3D representations for view synthesis and relightingcited by 0
  26. 2026EvalMVX: A Unified Benchmarking for Neural 3D Reconstruction under Diverse Multiview Setupscited by 0
  27. 2026WildRelight: A Real-World Benchmark and Physics-Guided Adaptation for Single-Image Relightingcited by 0
  28. 2026Wild3R: Feed-Forward 3D Gaussian Splatting from Unconstrained Sparse Photo Collectioncited by 0
  29. 2026Do Image Editing Models Understand Lighting?cited by 0
  30. 2026Dior: Drawing the Light of Image via Material-Decoupled Illumination Representationcited by 0