# PaletteNeRF: Palette-based Color Editing for NeRFs

arXiv: https://arxiv.org/abs/2212.12871 · HTML: https://mlsanity.com/paper/2212.12871/

## Key facts

- Cited by 18 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 49 references that are in the map.
- First posted on arXiv 25 Dec 2022.

## Builds on (most cited first, 25 of 25)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Instant Neural Graphics Primitives with a Multiresolution Hash Encoding](https://mlsanity.com/paper/2201.05989/) (2022)
- [CARLA: An Open Urban Driving Simulator](https://mlsanity.com/paper/1711.03938/) (2017)
- [D-NeRF: Neural Radiance Fields for Dynamic Scenes](https://mlsanity.com/paper/2011.13961/) (2020)
- [NeRF in the Wild: Neural Radiance Fields for Unconstrained Photo Collections](https://mlsanity.com/paper/2008.02268/) (2020)
- [Nerfies: Deformable Neural Radiance Fields](https://mlsanity.com/paper/2011.12948/) (2020)
- [Neural Sparse Voxel Fields](https://mlsanity.com/paper/2007.11571/) (2020)
- [PlenOctrees for Real-time Rendering of Neural Radiance Fields](https://mlsanity.com/paper/2103.14024/) (2021)
- [NeRF++: Analyzing and Improving Neural Radiance Fields](https://mlsanity.com/paper/2010.07492/) (2020)
- [GIRAFFE: Representing Scenes as Compositional Generative Neural Feature Fields](https://mlsanity.com/paper/2011.12100/) (2020)
- [KiloNeRF: Speeding up Neural Radiance Fields with Thousands of Tiny MLPs](https://mlsanity.com/paper/2103.13744/) (2021)
- [FastNeRF: High-Fidelity Neural Rendering at 200FPS](https://mlsanity.com/paper/2103.10380/) (2021)
- [Non-Rigid Neural Radiance Fields: Reconstruction and Novel View Synthesis of a Dynamic Scene From Monocular Video](https://mlsanity.com/paper/2012.12247/) (2020)
- [CLIP-NeRF: Text-and-Image Driven Manipulation of Neural Radiance Fields](https://mlsanity.com/paper/2112.05139/) (2021)
- [Advances in Neural Rendering](https://mlsanity.com/paper/2111.05849/) (2021)
- [Neural Scene Graphs for Dynamic Scenes](https://mlsanity.com/paper/2011.10379/) (2020)
- [Learning Object-Compositional Neural Radiance Field for Editable Scene Rendering](https://mlsanity.com/paper/2109.01847/) (2021)
- [Editing Conditional Radiance Fields](https://mlsanity.com/paper/2105.06466/) (2021)
- [Unsupervised Discovery of Object Radiance Fields](https://mlsanity.com/paper/2107.07905/) (2021)
- [Object-Centric Neural Scene Rendering](https://mlsanity.com/paper/2012.08503/) (2020)
- [Neural Volume Rendering: NeRF And Beyond](https://mlsanity.com/paper/2101.05204/) (2020)
- [LayerBuilder: Layer Decomposition for Interactive Image and Video Color Editing](https://mlsanity.com/paper/1701.03754/) (2017)
- [Pigmento: Pigment-Based Image Analysis and Editing](https://mlsanity.com/paper/1707.08323/) (2017)
- [Palette-based image decomposition, harmonization, and color transfer](https://mlsanity.com/paper/1804.01225/) (2018)
- [RGB Point Cloud Manipulation with Triangular Structures for Artistic Image Recoloring](https://mlsanity.com/paper/1912.04583/) (2019)

## Built on by (most cited first, 18 of 18)

- [Instruct-NeRF2NeRF: Editing 3D Scenes with Instructions](https://mlsanity.com/paper/2303.12789/) (2023)
- [GaussianEditor: Editing 3D Gaussians Delicately with Text Instructions](https://mlsanity.com/paper/2311.16037/) (2023)
- [ViCA-NeRF: View-Consistency-Aware 3D Editing of Neural Radiance Fields](https://mlsanity.com/paper/2402.00864/) (2024)
- [PIE-NeRF: Physics-based Interactive Elastodynamics with NeRF](https://mlsanity.com/paper/2311.13099/) (2023)
- [NeuManifold: Neural Watertight Manifold Reconstruction with Efficient and High-Quality Rendering Support](https://mlsanity.com/paper/2305.17134/) (2023)
- [Blended-NeRF: Zero-Shot Object Generation and Blending in Existing Neural Radiance Fields](https://mlsanity.com/paper/2306.12760/) (2023)
- [GenN2N: Generative NeRF2NeRF Translation](https://mlsanity.com/paper/2404.02788/) (2024)
- [TIGER: Text-Instructed 3D Gaussian Retrieval and Coherent Editing](https://mlsanity.com/paper/2405.14455/) (2024)
- [3DEgo: 3D Editing on the Go!](https://mlsanity.com/paper/2407.10102/) (2024)
- [VQ-NeRF: Neural Reflectance Decomposition and Editing with Vector Quantization](https://mlsanity.com/paper/2310.11864/) (2023)
- [Fast Sun-aligned Outdoor Scene Relighting based on TensoRF](https://mlsanity.com/paper/2311.03965/) (2023)
- [Diffusion-Based Attention Warping for Consistent 3D Scene Editing](https://mlsanity.com/paper/2412.07984/) (2024)
- [NeuBTF: Neural fields for BTF encoding and transfer](https://mlsanity.com/paper/2307.01199/) (2023)
- [LAENeRF: Local Appearance Editing for Neural Radiance Fields](https://mlsanity.com/paper/2312.09913/) (2023)
- [DATENeRF: Depth-Aware Text-based Editing of NeRFs](https://mlsanity.com/paper/2404.04526/) (2024)
- [3DGS-Drag: Dragging Gaussians for Intuitive Point-Based 3D Editing](https://mlsanity.com/paper/2601.07963/) (2026)
- [Text-guided Image-and-Shape Editing and Generation: A Short Survey](https://mlsanity.com/paper/2304.09244/) (2023)
- [Neural Poisson Solver: A Universal and Continuous Framework for Natural Signal Blending](https://mlsanity.com/paper/2407.08457/) (2024)

## Abstract (arXiv)

Neural Radiance Field (NeRF) is a powerful tool to faithfully generate novel views for scenes with only sparse captured images. Despite its strong capability for representing 3D scenes and their appearance, its editing ability is very limited. In this paper, we propose a simple but effective extension of vanilla NeRF, named PaletteNeRF, to enable efficient color editing on NeRF-represented scenes. Motivated by recent palette-based image decomposition works, we approximate each pixel color as a sum of palette colors modulated by additive weights. Instead of predicting pixel colors as in vanilla NeRFs, our method predicts additive weights. The underlying NeRF backbone could also be replaced with more recent NeRF models such as KiloNeRF to achieve real-time editing. Experimental results demonstrate that our method achieves efficient, view-consistent, and artifact-free color editing on a wide range of NeRF-represented scenes.