# PaletteNeRF: Palette-based Appearance Editing of Neural Radiance Fields

arXiv: https://arxiv.org/abs/2212.10699 · HTML: https://mlsanity.com/paper/2212.10699/

## Key facts

- Cited by 45 papers in the map (data through 14 Oct 2026).
- Builds on 32 of its 48 references that are in the map.
- First posted on arXiv 21 Dec 2022.

## Builds on (most cited first, 30 of 32)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Instant Neural Graphics Primitives with a Multiresolution Hash Encoding](https://mlsanity.com/paper/2201.05989/) (2022)
- [Mip-NeRF 360: Unbounded Anti-Aliased Neural Radiance Fields](https://mlsanity.com/paper/2111.12077/) (2021)
- [Arbitrary Style Transfer in Real-time with Adaptive Instance Normalization](https://mlsanity.com/paper/1703.06868/) (2017)
- [Plenoxels: Radiance Fields without Neural Networks](https://mlsanity.com/paper/2112.05131/) (2021)
- [TensoRF: Tensorial Radiance Fields](https://mlsanity.com/paper/2203.09517/) (2022)
- [NeRF in the Wild: Neural Radiance Fields for Unconstrained Photo Collections](https://mlsanity.com/paper/2008.02268/) (2020)
- [Volume Rendering of Neural Implicit Surfaces](https://mlsanity.com/paper/2106.12052/) (2021)
- [Direct Voxel Grid Optimization: Super-fast Convergence for Radiance Fields Reconstruction](https://mlsanity.com/paper/2111.11215/) (2021)
- [Scene Representation Networks: Continuous 3D-Structure-Aware Neural Scene Representations](https://mlsanity.com/paper/1906.01618/) (2019)
- [Language-driven Semantic Segmentation](https://mlsanity.com/paper/2201.03546/) (2022)
- [KiloNeRF: Speeding up Neural Radiance Fields with Thousands of Tiny MLPs](https://mlsanity.com/paper/2103.13744/) (2021)
- [Point-NeRF: Point-based Neural Radiance Fields](https://mlsanity.com/paper/2201.08845/) (2022)
- [NeRV: Neural Reflectance and Visibility Fields for Relighting and View Synthesis](https://mlsanity.com/paper/2012.03927/) (2020)
- [StyleNeRF: A Style-based 3D-Aware Generator for High-resolution Image Synthesis](https://mlsanity.com/paper/2110.08985/) (2021)
- [NeRD: Neural Reflectance Decomposition from Image Collections](https://mlsanity.com/paper/2012.03918/) (2020)
- [Decomposing NeRF for Editing via Feature Field Distillation](https://mlsanity.com/paper/2205.15585/) (2022)
- [CLIP-NeRF: Text-and-Image Driven Manipulation of Neural Radiance Fields](https://mlsanity.com/paper/2112.05139/) (2021)
- [PhySG: Inverse Rendering with Spherical Gaussians for Physics-based Material Editing and Relighting](https://mlsanity.com/paper/2104.00674/) (2021)
- [Neural-PIL: Neural Pre-Integrated Lighting for Reflectance Decomposition](https://mlsanity.com/paper/2110.14373/) (2021)
- [ARF: Artistic Radiance Fields](https://mlsanity.com/paper/2206.06360/) (2022)
- [NeRFactor: Neural Factorization of Shape and Reflectance Under an Unknown Illumination](https://mlsanity.com/paper/2106.01970/) (2021)
- [NeRFusion: Fusing Radiance Fields for Large-Scale Scene Reconstruction](https://mlsanity.com/paper/2203.11283/) (2022)
- [IntrinsicNeRF: Learning Intrinsic Neural Radiance Fields for Editable Novel View Synthesis](https://mlsanity.com/paper/2210.00647/) (2022)
- [UPST-NeRF: Universal Photorealistic Style Transfer of Neural Radiance Fields for 3D Scene](https://mlsanity.com/paper/2208.07059/) (2022)
- [NeROIC: Neural Rendering of Objects from Online Image Collections](https://mlsanity.com/paper/2201.02533/) (2022)
- [Comicolorization: Semi-Automatic Manga Colorization](https://mlsanity.com/paper/1706.06759/) (2017)
- [Real-Time Global Illumination Decomposition of Videos](https://mlsanity.com/paper/1908.01961/) (2019)

## Built on by (most cited first, 30 of 45)

- [DGE: Direct Gaussian 3D Editing by Consistent Multi-view Editing](https://mlsanity.com/paper/2404.18929/) (2024)
- [SKED: Sketch-guided Text-based 3D Editing](https://mlsanity.com/paper/2303.10735/) (2023)
- [Watch Your Steps: Local Image and Scene Editing by Text Instructions](https://mlsanity.com/paper/2308.08947/) (2023)
- [Reference-guided Controllable Inpainting of Neural Radiance Fields](https://mlsanity.com/paper/2304.09677/) (2023)
- [SHAP-EDITOR: Instruction-guided Latent 3D Editing in Seconds](https://mlsanity.com/paper/2312.09246/) (2023)
- [OR-NeRF: Object Removing from 3D Scenes Guided by Multiview Segmentation with Neural Radiance Fields](https://mlsanity.com/paper/2305.10503/) (2023)
- [RecolorNeRF: Layer Decomposed Radiance Fields for Efficient Color Editing of 3D Scenes](https://mlsanity.com/paper/2301.07958/) (2023)
- [NeuManifold: Neural Watertight Manifold Reconstruction with Efficient and High-Quality Rendering Support](https://mlsanity.com/paper/2305.17134/) (2023)
- [RelitLRM: Generative Relightable Radiance for Large Reconstruction Models](https://mlsanity.com/paper/2410.06231/) (2024)
- [ED-NeRF: Efficient Text-Guided Editing of 3D Scene with Latent Space NeRF](https://mlsanity.com/paper/2310.02712/) (2023)
- [Seal-3D: Interactive Pixel-Level Editing for Neural Radiance Fields](https://mlsanity.com/paper/2307.15131/) (2023)
- [ProteusNeRF: Fast Lightweight NeRF Editing using 3D-Aware Image Context](https://mlsanity.com/paper/2310.09965/) (2023)
- [LatentEditor: Text Driven Local Editing of 3D Scenes](https://mlsanity.com/paper/2312.09313/) (2023)
- [Neural Radiance Field-based Visual Rendering: A Comprehensive Review](https://mlsanity.com/paper/2404.00714/) (2024)
- [Neural Impostor: Editing Neural Radiance Fields with Explicit Shape Manipulation](https://mlsanity.com/paper/2310.05391/) (2023)
- [NeRF Analogies: Example-Based Visual Attribute Transfer for NeRFs](https://mlsanity.com/paper/2402.08622/) (2024)
- [GenN2N: Generative NeRF2NeRF Translation](https://mlsanity.com/paper/2404.02788/) (2024)
- [CodecNeRF: Toward Fast Encoding and Decoding, Compact, and High-quality Novel-view Synthesis](https://mlsanity.com/paper/2404.04913/) (2024)
- [Query-based Semantic Gaussian Field for Scene Representation in Reinforcement Learning](https://mlsanity.com/paper/2406.02370/) (2024)
- [GeometrySticker: Enabling Ownership Claim of Recolorized Neural Radiance Fields](https://mlsanity.com/paper/2407.13390/) (2024)
- [Geometry-Aware Diffusion Models for Multiview Scene Inpainting](https://mlsanity.com/paper/2502.13335/) (2025)
- [LAENeRF: Local Appearance Editing for Neural Radiance Fields](https://mlsanity.com/paper/2312.09913/) (2023)
- [Freditor: High-Fidelity and Transferable NeRF Editing by Frequency Decomposition](https://mlsanity.com/paper/2404.02514/) (2024)
- [DATENeRF: Depth-Aware Text-based Editing of NeRFs](https://mlsanity.com/paper/2404.04526/) (2024)
- [Low-Frequency First: Eliminating Floating Artifacts in 3D Gaussian Splatting](https://mlsanity.com/paper/2508.02493/) (2025)
- [Parameterization-driven Neural Surface Reconstruction for Object-oriented Editing in Neural Rendering](https://mlsanity.com/paper/2310.05524/) (2023)
- [SealD-NeRF: Interactive Pixel-Level Editing for Dynamic Scenes by Neural Radiance Fields](https://mlsanity.com/paper/2402.13510/) (2024)
- [VULCAN: Tool-Augmented Multi Agents for Iterative 3D Object Arrangement](https://mlsanity.com/paper/2512.22351/) (2025)
- [Text-guided Image-and-Shape Editing and Generation: A Short Survey](https://mlsanity.com/paper/2304.09244/) (2023)
- [SplatMesh: Interactive 3D Segmentation and Editing Using Mesh-Based Gaussian Splatting](https://mlsanity.com/paper/2312.15856/) (2023)

## Abstract (arXiv)

Recent advances in neural radiance fields have enabled the high-fidelity 3D reconstruction of complex scenes for novel view synthesis. However, it remains underexplored how the appearance of such representations can be efficiently edited while maintaining photorealism. In this work, we present PaletteNeRF, a novel method for photorealistic appearance editing of neural radiance fields (NeRF) based on 3D color decomposition. Our method decomposes the appearance of each 3D point into a linear combination of palette-based bases (i.e., 3D segmentations defined by a group of NeRF-type functions) that are shared across the scene. While our palette-based bases are view-independent, we also predict a view-dependent function to capture the color residual (e.g., specular shading). During training, we jointly optimize the basis functions and the color palettes, and we also introduce novel regularizers to encourage the spatial coherence of the decomposition. Our method allows users to efficiently edit the appearance of the 3D scene by modifying the color palettes. We also extend our framework with compressed semantic features for semantic-aware appearance editing. We demonstrate that our technique is superior to baseline methods both quantitatively and qualitatively for appearance editing of complex real-world scenes.