# Bilateral Guided Radiance Field Processing

arXiv: https://arxiv.org/abs/2406.00448 · HTML: https://mlsanity.com/paper/2406.00448/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- Builds on 28 of its 60 references that are in the map.
- First posted on arXiv 1 Jun 2024.

## Builds on (most cited first, 28 of 28)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [Instant Neural Graphics Primitives with a Multiresolution Hash Encoding](https://mlsanity.com/paper/2201.05989/) (2022)
- [InstructPix2Pix: Learning to Follow Image Editing Instructions](https://mlsanity.com/paper/2211.09800/) (2022)
- [Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains](https://mlsanity.com/paper/2006.10739/) (2020)
- [Restormer: Efficient Transformer for High-Resolution Image Restoration](https://mlsanity.com/paper/2111.09881/) (2021)
- [NeRF in the Wild: Neural Radiance Fields for Unconstrained Photo Collections](https://mlsanity.com/paper/2008.02268/) (2020)
- [Nerfies: Deformable Neural Radiance Fields](https://mlsanity.com/paper/2011.12948/) (2020)
- [NeRV: Neural Reflectance and Visibility Fields for Relighting and View Synthesis](https://mlsanity.com/paper/2012.03927/) (2020)
- [Instruct-NeRF2NeRF: Editing 3D Scenes with Instructions](https://mlsanity.com/paper/2303.12789/) (2023)
- [Decomposing NeRF for Editing via Feature Field Distillation](https://mlsanity.com/paper/2205.15585/) (2022)
- [Extracting Triangular 3D Models, Materials, and Lighting From Images](https://mlsanity.com/paper/2111.12503/) (2021)
- [NeRF in the Dark: High Dynamic Range View Synthesis from Noisy Raw Images](https://mlsanity.com/paper/2111.13679/) (2021)
- [Unprocessing Images for Learned Raw Denoising](https://mlsanity.com/paper/1811.11127/) (2018)
- [Editing Conditional Radiance Fields](https://mlsanity.com/paper/2105.06466/) (2021)
- [Deep Bilateral Learning for Real-Time Image Enhancement](https://mlsanity.com/paper/1707.02880/) (2017)
- [CodeNeRF: Disentangled Neural Radiance Fields for Object Categories](https://mlsanity.com/paper/2109.01750/) (2021)
- [Optimizing the Latent Space of Generative Networks](https://mlsanity.com/paper/1707.05776/) (2017)
- [RobustNeRF: Ignoring Distractors with Robust Losses](https://mlsanity.com/paper/2302.00833/) (2023)
- [SAMURAI: Shape And Material from Unconstrained Real-world Arbitrary Image collections](https://mlsanity.com/paper/2205.15768/) (2022)
- [The Fast Bilateral Solver](https://mlsanity.com/paper/1511.03296/) (2015)
- [Lighting up NeRF via Unsupervised Decomposition and Enhancement](https://mlsanity.com/paper/2307.10664/) (2023)
- [ReconfigISP: Reconfigurable Camera Image Processing Pipeline](https://mlsanity.com/paper/2109.04760/) (2021)
- [RecolorNeRF: Layer Decomposed Radiance Fields for Efficient Color Editing of 3D Scenes](https://mlsanity.com/paper/2301.07958/) (2023)
- [Learning sRGB-to-Raw-RGB De-rendering with Content-Aware Metadata](https://mlsanity.com/paper/2206.01813/) (2022)
- [Automatic ISP image quality tuning using non-linear optimization](https://mlsanity.com/paper/1902.09023/) (2019)
- [SeamlessNeRF: Stitching Part NeRFs with Gradient Propagation](https://mlsanity.com/paper/2311.16127/) (2023)

## Built on by (most cited first, 28 of 28)

- [Luminance-GS: Adapting 3D Gaussian Splatting to Challenging Lighting Conditions with View-Adaptive Curve Adjustment](https://mlsanity.com/paper/2504.01503/) (2025)
- [RealX3D: A Physically-Degraded 3D Benchmark for Multi-view Visual Restoration and Reconstruction](https://mlsanity.com/paper/2512.23437/) (2025)
- [RecGS: Removing Water Caustic with Recurrent Gaussian Splatting](https://mlsanity.com/paper/2407.10318/) (2024)
- [Unifying Appearance Codes and Bilateral Grids for Driving Scene Gaussian Splatting](https://mlsanity.com/paper/2506.05280/) (2025)
- [GaSLight: Gaussian Splats for Spatially-Varying Lighting in HDR](https://mlsanity.com/paper/2504.10809/) (2025)
- [Unifying Color and Lightness Correction with View-Adaptive Curve Adjustment for Robust 3D Novel View Synthesis](https://mlsanity.com/paper/2602.18322/) (2026)
- [PPISP: Physically-Plausible Compensation and Control of Photometric Variations in Radiance Field Reconstruction](https://mlsanity.com/paper/2601.18336/) (2026)
- [Martian World Model: Controllable Video Synthesis with Physically Accurate 3D Reconstructions](https://mlsanity.com/paper/2507.07978/) (2025)
- [CHROMA: Consistent Harmonization of Multi-View Appearance via Bilateral Grid Prediction](https://mlsanity.com/paper/2507.15748/) (2025)
- [Learning Neural Exposure Fields for View Synthesis](https://mlsanity.com/paper/2510.08279/) (2025)
- [SimULi: Real-Time LiDAR and Camera Simulation with Unscented Transforms](https://mlsanity.com/paper/2510.12901/) (2025)
- [Confidence-Based Mesh Extraction from 3D Gaussians](https://mlsanity.com/paper/2603.24725/) (2026)
- [RAWild: Sensor-Agnostic RAW Object Detection via Physics-Guided Curve and Grid Modeling](https://mlsanity.com/paper/2605.05941/) (2026)
- [UniVerse: Unleashing the Scene Prior of Video Diffusion Models for Robust Radiance Field Reconstruction](https://mlsanity.com/paper/2510.01669/) (2025)
- [DC4GS: Directional Consistency-Driven Adaptive Density Control for 3D Gaussian Splatting](https://mlsanity.com/paper/2510.26921/) (2025)
- [GlowGS: Generative Semantic Feature Learning for 3D Gaussian Splatting in Nighttime Glow Scenes](https://mlsanity.com/paper/2605.23602/) (2026)
- [Hybrid Rendering for Multimodal Autonomous Driving: Merging Neural and Physics-Based Simulation](https://mlsanity.com/paper/2503.09464/) (2025)
- [A Constrained Optimization Approach for Gaussian Splatting from Coarsely-posed Images and Noisy Lidar Point Clouds](https://mlsanity.com/paper/2504.09129/) (2025)
- [Lumos3D: A Single-Forward Framework for Low-Light 3D Scene Restoration](https://mlsanity.com/paper/2511.09818/) (2025)
- [ReCamDriving: LiDAR-Free Camera-Controlled Video Synthesis for Novel Trajectories](https://mlsanity.com/paper/2512.03621/) (2025)
- [Realizing Immersive Volumetric Video: A Multimodal Framework for 6-DoF VR Engagement](https://mlsanity.com/paper/2604.09473/) (2026)
- [FreeScale: Scaling 3D Scenes via Certainty-Aware Free-View Generation](https://mlsanity.com/paper/2604.10512/) (2026)
- [Light 'em Up: Enabling Few-Shot Low-Light 3D Gaussian Splatting with Multi-Scale Explicit Retinex Illumination Decoupling](https://mlsanity.com/paper/2604.24053/) (2026)
- [P2GS: Physical Prior-guided Gaussian Splatting for Photometrically Consistent Urban Reconstruction](https://mlsanity.com/paper/2605.16925/) (2026)
- [Log NeRF: Comparing Spaces for Learning Radiance Fields](https://mlsanity.com/paper/2512.09375/) (2025)
- [Aes3D: Aesthetic Assessment in 3D Gaussian Splatting](https://mlsanity.com/paper/2605.05155/) (2026)
- [FujinSplat: Seeing Through Smoke with RAW-Domain Gaussian Splatting](https://mlsanity.com/paper/2609.06017/) (2026)
- [EvenSplat: Coupled 2D-3D Decomposition for Gaussian Splatting under Exposure and Illumination Variation](https://mlsanity.com/paper/2610.01876/) (2026)

## Abstract (arXiv)

Neural Radiance Fields (NeRF) achieves unprecedented performance in synthesizing novel view synthesis, utilizing multi-view consistency. When capturing multiple inputs, image signal processing (ISP) in modern cameras will independently enhance them, including exposure adjustment, color correction, local tone mapping, etc. While these processings greatly improve image quality, they often break the multi-view consistency assumption, leading to "floaters" in the reconstructed radiance fields. To address this concern without compromising visual aesthetics, we aim to first disentangle the enhancement by ISP at the NeRF training stage and re-apply user-desired enhancements to the reconstructed radiance fields at the finishing stage. Furthermore, to make the re-applied enhancements consistent between novel views, we need to perform imaging signal processing in 3D space (i.e. "3D ISP"). For this goal, we adopt the bilateral grid, a locally-affine model, as a generalized representation of ISP processing. Specifically, we optimize per-view 3D bilateral grids with radiance fields to approximate the effects of camera pipelines for each input view. To achieve user-adjustable 3D finishing, we propose to learn a low-rank 4D bilateral grid from a given single view edit, lifting photo enhancements to the whole 3D scene. We demonstrate our approach can boost the visual quality of novel view synthesis by effectively removing floaters and performing enhancements from user retouching. The source code and our data are available at: https://bilarfpro.github.io.