# Knowledge Distillation for Continual Learning of Biomedical Neural Fields

arXiv: https://arxiv.org/abs/2511.21409 · HTML: https://mlsanity.com/paper/2511.21409/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in Distillation for continual learning, one of 12 directions in Knowledge distillation.
- Builds on 14 of its 18 references that are in the map.
- First posted on arXiv 26 Nov 2025.

## Placed in

- Knowledge distillation / [Distillation for continual learning](https://mlsanity.com/distill/distillation-for-continual-learning/)

## Builds on (most cited first, 14 of 14)

- [Overcoming catastrophic forgetting in neural networks](https://mlsanity.com/paper/1612.00796/) (2016)
- [Implicit Neural Representations with Periodic Activation Functions](https://mlsanity.com/paper/2006.09661/) (2020)
- [On the Spectral Bias of Neural Networks](https://mlsanity.com/paper/1806.08734/) (2018)
- [A Comprehensive Survey of Continual Learning: Theory, Method and Application](https://mlsanity.com/paper/2302.00487/) (2023)
- [An Empirical Investigation of Catastrophic Forgetting in Gradient-Based Neural Networks](https://mlsanity.com/paper/1312.6211/) (2013)
- [NeRP: Implicit Neural Representation Learning with Prior Embedding for Sparsely Sampled Image Reconstruction](https://mlsanity.com/paper/2108.10991/) (2021)
- [A Theoretical Analysis of Catastrophic Forgetting through the NTK Overlap Matrix](https://mlsanity.com/paper/2010.04003/) (2020)
- [Implicit Neural Representation in Medical Imaging: A Comparative Survey](https://mlsanity.com/paper/2307.16142/) (2023)
- [FINER: Flexible spectral-bias tuning in Implicit NEural Representation by Variable-periodic Activation Functions](https://mlsanity.com/paper/2312.02434/) (2023)
- [Dynamic CT Reconstruction from Limited Views with Implicit Neural Representations and Parametric Motion Fields](https://mlsanity.com/paper/2104.11745/) (2021)
- [Disorder-invariant Implicit Neural Representation](https://mlsanity.com/paper/2304.00837/) (2023)
- [CLNeRF: Continual Learning Meets NeRF](https://mlsanity.com/paper/2308.14816/) (2023)
- [MEIL-NeRF: Memory-Efficient Incremental Learning of Neural Radiance Fields](https://mlsanity.com/paper/2212.08328/) (2022)
- [Fast Medical Shape Reconstruction via Meta-learned Implicit Neural Representations](https://mlsanity.com/paper/2409.07100/) (2024)

## Built on by (most cited first, 2 of 2)

- [Continual Learning as a Multiphase Moving-Boundary Problem](https://mlsanity.com/paper/2606.01863/) (2026)
- [Distilled Roads: Generalisable Road Network Extraction Across Sensors, Resolutions, and Region](https://mlsanity.com/paper/2608.03407/) (2026)

## Abstract (arXiv)

Neural fields are increasingly used as a light-weight, continuous, and differentiable signal representation in (bio)medical imaging. However, unlike discrete signal representations such as voxel grids, neural fields cannot be easily extended. As neural fields are, in essence, neural networks, prior signals represented in a neural field will degrade when the model is presented with new data due to catastrophic forgetting. This work examines the extent to which different neural field approaches suffer from catastrophic forgetting and proposes a strategy to mitigate this issue. We consider the scenario in which data becomes available incrementally, with only the most recent data available for neural field fitting. In a series of experiments on cardiac cine MRI data, we demonstrate how knowledge distillation mitigates catastrophic forgetting when the spatiotemporal domain is enlarged or the dimensionality of the represented signal is increased. We find that the amount of catastrophic forgetting depends, to a large extent, on the neural fields model used, and that distillation could enable continual learning in neural fields.