# CLNeRF: Continual Learning Meets NeRF

arXiv: https://arxiv.org/abs/2308.14816 · HTML: https://mlsanity.com/paper/2308.14816/

## Key facts

- Cited by 24 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 33 references that are in the map.
- First posted on arXiv 28 Aug 2023.

## Builds on (most cited first, 18 of 18)

- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [Overcoming catastrophic forgetting in neural networks](https://mlsanity.com/paper/1612.00796/) (2016)
- [Instant Neural Graphics Primitives with a Multiresolution Hash Encoding](https://mlsanity.com/paper/2201.05989/) (2022)
- [Learning without Forgetting](https://mlsanity.com/paper/1606.09282/) (2016)
- [Gradient Episodic Memory for Continual Learning](https://mlsanity.com/paper/1706.08840/) (2017)
- [NeRF in the Wild: Neural Radiance Fields for Unconstrained Photo Collections](https://mlsanity.com/paper/2008.02268/) (2020)
- [Continual Learning with Deep Generative Replay](https://mlsanity.com/paper/1705.08690/) (2017)
- [Prioritized Experience Replay](https://mlsanity.com/paper/1511.05952/) (2015)
- [Direct Voxel Grid Optimization: Super-fast Convergence for Radiance Fields Reconstruction](https://mlsanity.com/paper/2111.11215/) (2021)
- [Neural Sparse Voxel Fields](https://mlsanity.com/paper/2007.11571/) (2020)
- [PackNet: Adding Multiple Tasks to a Single Network by Iterative Pruning](https://mlsanity.com/paper/1711.05769/) (2017)
- [NeRF++: Analyzing and Improving Neural Radiance Fields](https://mlsanity.com/paper/2010.07492/) (2020)
- [Block-NeRF: Scalable Large Scene Neural View Synthesis](https://mlsanity.com/paper/2202.05263/) (2022)
- [KiloNeRF: Speeding up Neural Radiance Fields with Thousands of Tiny MLPs](https://mlsanity.com/paper/2103.13744/) (2021)
- [The CLEAR Benchmark: Continual LEArning on Real-World Imagery](https://mlsanity.com/paper/2201.06289/) (2022)
- [NerfAcc: A General NeRF Acceleration Toolbox](https://mlsanity.com/paper/2210.04847/) (2022)
- [Online Continual Learning with Natural Distribution Shifts: An Empirical Study with Visual Data](https://mlsanity.com/paper/2108.09020/) (2021)
- [MEIL-NeRF: Memory-Efficient Incremental Learning of Neural Radiance Fields](https://mlsanity.com/paper/2212.08328/) (2022)

## Built on by (most cited first, 24 of 24)

- [NARUTO: Neural Active Reconstruction from Uncertain Target Observations](https://mlsanity.com/paper/2402.18771/) (2024)
- [GaussianUpdate: Continual 3D Gaussian Splatting Update for Changing Environments](https://mlsanity.com/paper/2508.08867/) (2025)
- [CL-Splats: Continual Learning of Gaussian Splatting with Local Optimization](https://mlsanity.com/paper/2506.21117/) (2025)
- [SCARF: Scalable Continual Learning Framework for Memory-efficient Multiple Neural Radiance Fields](https://mlsanity.com/paper/2409.04482/) (2024)
- [Variational Bayes Gaussian Splatting](https://mlsanity.com/paper/2410.03592/) (2024)
- [Multi-Layer Gaussian Splatting for Immersive Anatomy Visualization](https://mlsanity.com/paper/2410.16978/) (2024)
- [IL-NeRF: Incremental Learning for Neural Radiance Fields with Camera Pose Alignment](https://mlsanity.com/paper/2312.05748/) (2023)
- [Understanding while Exploring: Semantics-driven Active Mapping](https://mlsanity.com/paper/2506.00225/) (2025)
- [Learn to Memorize and to Forget: A Continual Learning Perspective of Dynamic SLAM](https://mlsanity.com/paper/2407.13338/) (2024)
- [Changes in Real Time: Online Scene Change Detection with Multi-View Fusion](https://mlsanity.com/paper/2511.12370/) (2025)
- [3D Scene Change Modeling With Consistent Multi-View Aggregation](https://mlsanity.com/paper/2512.22830/) (2025)
- [CD-NGP: A Fast Scalable Continual Representation for Dynamic Scenes](https://mlsanity.com/paper/2409.05166/) (2024)
- [Tile and Slide : A New Framework for Scaling NeRF from Local to Global 3D Earth Observation](https://mlsanity.com/paper/2507.01631/) (2025)
- [Knowledge Distillation for Continual Learning of Biomedical Neural Fields](https://mlsanity.com/paper/2511.21409/) (2025)
- [Gaussian Time Machine: A Real-Time Rendering Methodology for Time-Variant Appearances](https://mlsanity.com/paper/2405.13694/) (2024)
- [Task-agnostic Lifelong Robot Learning with Retrieval-based Weighted Local Adaptation](https://mlsanity.com/paper/2410.02995/) (2024)
- [LTGS: Long-Term Gaussian Scene Chronology From Sparse View Updates](https://mlsanity.com/paper/2510.09881/) (2025)
- [ChronoGS: Disentangling Invariants and Changes in Multi-Period Scenes](https://mlsanity.com/paper/2511.18794/) (2025)
- [RecurGS: Interactive Scene Modeling via Discrete-State Recurrent Gaussian Fusion](https://mlsanity.com/paper/2512.18386/) (2025)
- [Improving Continual Learning for Gaussian Splatting based Environments Reconstruction on Commercial Off-the-Shelf Edge Devices](https://mlsanity.com/paper/2603.08499/) (2026)
- [DISORF: A Distributed Online 3D Reconstruction Framework for Mobile Robots](https://mlsanity.com/paper/2403.00228/) (2024)
- [Fully-fused Multi-Layer Perceptrons on Intel Data Center GPUs](https://mlsanity.com/paper/2403.17607/) (2024)
- [Incremental Multi-Scene Modeling via Continual Neural Graphics Primitives](https://mlsanity.com/paper/2411.19903/) (2024)
- [Planetary Feature Fields are Scalable Earth Representations](https://mlsanity.com/paper/2609.37784/) (2026)

## Abstract (arXiv)

Novel view synthesis aims to render unseen views given a set of calibrated images. In practical applications, the coverage, appearance or geometry of the scene may change over time, with new images continuously being captured. Efficiently incorporating such continuous change is an open challenge. Standard NeRF benchmarks only involve scene coverage expansion. To study other practical scene changes, we propose a new dataset, World Across Time (WAT), consisting of scenes that change in appearance and geometry over time. We also propose a simple yet effective method, CLNeRF, which introduces continual learning (CL) to Neural Radiance Fields (NeRFs). CLNeRF combines generative replay and the Instant Neural Graphics Primitives (NGP) architecture to effectively prevent catastrophic forgetting and efficiently update the model when new data arrives. We also add trainable appearance and geometry embeddings to NGP, allowing a single compact model to handle complex scene changes. Without the need to store historical images, CLNeRF trained sequentially over multiple scans of a changing scene performs on-par with the upper bound model trained on all scans at once. Compared to other CL baselines CLNeRF performs much better across standard benchmarks and WAT. The source code, and the WAT dataset are available at https://github.com/IntelLabs/CLNeRF. Video presentation is available at: https://youtu.be/nLRt6OoDGq0?si=8yD6k-8MMBJInQPs