# Learning to Restore More: Continual Capability Expansion for Pretrained Image Restoration Models

arXiv: https://arxiv.org/abs/2608.30305 · HTML: https://mlsanity.com/paper/2608.30305/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 32 of its 61 references that are in the map.
- First posted on arXiv 31 Aug 2026.

## Builds on (most cited first, 30 of 32)

- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [SGDR: Stochastic Gradient Descent with Warm Restarts](https://mlsanity.com/paper/1608.03983/) (2016)
- [Learning without Forgetting](https://mlsanity.com/paper/1606.09282/) (2016)
- [iCaRL: Incremental Classifier and Representation Learning](https://mlsanity.com/paper/1611.07725/) (2016)
- [Gradient Episodic Memory for Continual Learning](https://mlsanity.com/paper/1706.08840/) (2017)
- [Enhanced Deep Residual Networks for Single Image Super-Resolution](https://mlsanity.com/paper/1707.02921/) (2017)
- [Deep Multi-scale Convolutional Neural Network for Dynamic Scene Deblurring](https://mlsanity.com/paper/1612.02177/) (2016)
- [Benchmarking Single Image Dehazing and Beyond](https://mlsanity.com/paper/1712.04143/) (2017)
- [NIMA: Neural Image Assessment](https://mlsanity.com/paper/1709.05424/) (2017)
- [Deep Joint Rain Detection and Removal from a Single Image](https://mlsanity.com/paper/1609.07769/) (2016)
- [Density-aware Single Image De-raining using a Multi-stream Dense Network](https://mlsanity.com/paper/1802.07412/) (2018)
- [DesnowNet: Context-Aware Deep Network for Snow Removal](https://mlsanity.com/paper/1708.04512/) (2017)
- [Boosting Continual Learning of Vision-Language Models via Mixture-of-Experts Adapters](https://mlsanity.com/paper/2403.11549/) (2024)
- [Image De-raining Using a Conditional Generative Adversarial Network](https://mlsanity.com/paper/1701.05957/) (2017)
- [AdaIR: Adaptive All-in-One Image Restoration via Frequency Mining and Modulation](https://mlsanity.com/paper/2403.14614/) (2024)
- [AutoDIR: Automatic All-in-One Image Restoration with Latent Diffusion](https://mlsanity.com/paper/2310.10123/) (2023)
- [MambaIRv2: Attentive State Space Restoration](https://mlsanity.com/paper/2411.15269/) (2024)
- [Self-Expansion of Pre-trained Models with Mixture of Adapters for Continual Learning](https://mlsanity.com/paper/2403.18886/) (2024)
- [Toward Real-world Single Image Deraining: A New Benchmark and Beyond](https://mlsanity.com/paper/2206.05514/) (2022)
- [Vision-Language Gradient Descent-driven All-in-One Deep Unfolding Networks](https://mlsanity.com/paper/2503.16930/) (2025)
- [Visual-Instructed Degradation Diffusion for All-in-One Image Restoration](https://mlsanity.com/paper/2506.16960/) (2025)
- [Learning Enriched Features via Selective State Spaces Model for Efficient Image Deblurring](https://mlsanity.com/paper/2403.20106/) (2024)
- [Prompt-based Ingredient-Oriented All-in-One Image Restoration](https://mlsanity.com/paper/2309.03063/) (2023)
- [Learning Continuous Wasserstein Barycenter Space for Generalized All-in-One Image Restoration](https://mlsanity.com/paper/2602.23169/) (2026)
- [Mixed Hierarchy Network for Image Restoration](https://mlsanity.com/paper/2302.09554/) (2023)
- [Continual All-in-One Adverse Weather Removal with Knowledge Replay on a Unified Network Structure](https://mlsanity.com/paper/2403.07292/) (2024)
- [Prior-guided Hierarchical Harmonization Network for Efficient Image Dehazing](https://mlsanity.com/paper/2503.01136/) (2025)
- [Unpaired Photo-realistic Image Deraining with Energy-informed Diffusion Model](https://mlsanity.com/paper/2407.17193/) (2024)
- [Learning Physics-Informed Noise Models from Dark Frames for Low-Light Raw Image Denoising](https://mlsanity.com/paper/2310.09126/) (2023)
- [Emphasizing Crucial Features for Efficient Image Restoration](https://mlsanity.com/paper/2405.11468/) (2024)

## Abstract (arXiv)

Image restoration models are typically trained with a fixed set of capabilities. When new restoration requirements emerge, existing solutions usually train additional models or jointly retrain the original model with both new and historical data. Instead of designing another restoration backbone, we investigate how a trained restorer can continually acquire new capabilities without forgetting those learned previously. We propose RestoreMore, a continual capability-expansion framework that preserves the pretrained restoration model as a frozen capability anchor and learns residual expansion modules for newly arriving degradations. RestoreMore introduces a capability-oriented bi-level routing mechanism at multiple feature stages. The first routing level identifies restoration capabilities relevant to the current input, while the second selects and combines a sparse set of complementary degradation experts. This design enables newly introduced tasks to selectively reuse historical restoration knowledge and progressively enriches the expert bank available for subsequent restoration tasks. Extensive experiments on a wide range of restoration benchmarks demonstrate that RestoreMore consistently acquires new restoration abilities while preserving and improving previously learned capabilities.