# Towards Effective Multiple-in-One Image Restoration: A Sequential and Prompt Learning Strategy

arXiv: https://arxiv.org/abs/2401.03379 · HTML: https://mlsanity.com/paper/2401.03379/

## Key facts

- Cited by 50 papers in the map (data through 14 Oct 2026).
- Builds on 15 of its 68 references that are in the map.
- First posted on arXiv 7 Jan 2024.

## Builds on (most cited first, 15 of 15)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [Conditional Generative Adversarial Nets](https://mlsanity.com/paper/1411.1784/) (2014)
- [Decoupled Weight Decay Regularization](https://mlsanity.com/paper/1711.05101/) (2017)
- [Continual Learning with Deep Generative Replay](https://mlsanity.com/paper/1705.08690/) (2017)
- [Deep Retinex Decomposition for Low-Light Enhancement](https://mlsanity.com/paper/1808.04560/) (2018)
- [Vision Transformers for Single Image Dehazing](https://mlsanity.com/paper/2204.03883/) (2022)
- [Visual Prompting via Image Inpainting](https://mlsanity.com/paper/2209.00647/) (2022)
- [PromptIR: Prompting for All-in-One Blind Image Restoration](https://mlsanity.com/paper/2306.13090/) (2023)
- [ProRes: Exploring Degradation-aware Visual Prompt for Universal Image Restoration](https://mlsanity.com/paper/2306.13653/) (2023)
- [HAT: Hybrid Attention Transformer for Image Restoration](https://mlsanity.com/paper/2309.05239/) (2023)
- [On Efficient Transformer-Based Image Pre-training for Low-Level Vision](https://mlsanity.com/paper/2112.10175/) (2021)
- [Unifying Image Processing as Visual Prompting Question Answering](https://mlsanity.com/paper/2310.10513/) (2023)
- [Discovering Distinctive "Semantics" in Super-Resolution Networks](https://mlsanity.com/paper/2108.00406/) (2021)

## Built on by (most cited first, 30 of 50)

- [A Survey on All-in-One Image Restoration: Taxonomy, Evaluation and Future Trends](https://mlsanity.com/paper/2410.15067/) (2024)
- [RestoreAgent: Autonomous Image Restoration Agent via Multimodal Large Language Models](https://mlsanity.com/paper/2407.18035/) (2024)
- [An Intelligent Agentic System for Complex Image Restoration Problems](https://mlsanity.com/paper/2410.17809/) (2024)
- [FoundIR: Unleashing Million-scale Training Data to Advance Foundation Models for Image Restoration](https://mlsanity.com/paper/2412.01427/) (2024)
- [Restore-RWKV: Efficient and Effective Medical Image Restoration with RWKV](https://mlsanity.com/paper/2407.11087/) (2024)
- [JarvisIR: Elevating Autonomous Driving Perception with Intelligent Image Restoration](https://mlsanity.com/paper/2504.04158/) (2025)
- [All-In-One Medical Image Restoration via Task-Adaptive Routing](https://mlsanity.com/paper/2405.19769/) (2024)
- [A Coreset Selection of Coreset Selection Literature: Introduction and Recent Advances](https://mlsanity.com/paper/2505.17799/) (2025)
- [Multi-Agent Image Restoration](https://mlsanity.com/paper/2503.09403/) (2025)
- [Multi-Expert Adaptive Selection: Task-Balancing for All-in-One Image Restoration](https://mlsanity.com/paper/2407.19139/) (2024)
- [UniRestorer: Universal Image Restoration via Adaptively Estimating Image Degradation at Proper Granularity](https://mlsanity.com/paper/2412.20157/) (2024)
- [Instruct-IPT: All-in-One Image Processing Transformer via Weight Modulation](https://mlsanity.com/paper/2407.00676/) (2024)
- [FoundIR-v2: Optimizing Pre-Training Data Mixtures for Image Restoration Foundation Model](https://mlsanity.com/paper/2512.09282/) (2025)
- [UCIP: A Universal Framework for Compressed Image Super-Resolution using Dynamic Prompt](https://mlsanity.com/paper/2407.13108/) (2024)
- [Interpreting Low-level Vision Models with Causal Effect Maps](https://mlsanity.com/paper/2407.19789/) (2024)
- [Learning Domain-Aware Task Prompt Representations for Multi-Domain All-in-One Image Restoration](https://mlsanity.com/paper/2603.01725/) (2026)
- [A Preliminary Exploration Towards General Image Restoration](https://mlsanity.com/paper/2408.15143/) (2024)
- [Beyond Degradation Redundancy: Contrastive Prompt Learning for All-in-One Image Restoration](https://mlsanity.com/paper/2504.09973/) (2025)
- [Restore-R1: Efficient Image Restoration Agents via Reinforcement Learning with Multimodal LLM Perceptual Feedback](https://mlsanity.com/paper/2512.18599/) (2025)
- [Training-Free Large Model Priors for Multiple-in-One Image Restoration](https://mlsanity.com/paper/2407.13181/) (2024)
- [UARE: A Unified Vision-Language Model for Image Quality Assessment, Restoration, and Enhancement](https://mlsanity.com/paper/2512.06750/) (2025)
- [TIR-Agent: Training an Explorative and Efficient Agent for Image Restoration](https://mlsanity.com/paper/2603.27742/) (2026)
- [TPGDiff: Hierarchical Triple-Prior Guided Diffusion for Image Restoration](https://mlsanity.com/paper/2601.20306/) (2026)
- [OPERA: An Agent for Image Restoration with End-to-End Joint Planning-Execution Optimization](https://mlsanity.com/paper/2605.22104/) (2026)
- [Self-Evolving Agentic Image Restoration via Deliberate Planning and Intuitive Execution](https://mlsanity.com/paper/2606.28971/) (2026)
- [A Progressive Image Restoration Network for High-order Degradation Imaging in Remote Sensing](https://mlsanity.com/paper/2412.07195/) (2024)
- [Latent Harmony: Synergistic Unified UHD Image Restoration via Latent Space Regularization and Controllable Refinement](https://mlsanity.com/paper/2510.07961/) (2025)
- [Enhancing Infrared Vision: Progressive Prompt Fusion Network and Benchmark](https://mlsanity.com/paper/2510.09343/) (2025)
- [Restore, Assess, Repeat: A Unified Framework for Iterative Image Restoration](https://mlsanity.com/paper/2603.26385/) (2026)
- [UPCMR: A Universal Prompt-guided Model for Random Sampling Cardiac MRI Reconstruction](https://mlsanity.com/paper/2502.14899/) (2025)

## Abstract (arXiv)

While single task image restoration (IR) has achieved significant successes, it remains a challenging issue to train a single model which can tackle multiple IR tasks. In this work, we investigate in-depth the multiple-in-one (MiO) IR problem, which comprises seven popular IR tasks. We point out that MiO IR faces two pivotal challenges: the optimization of diverse objectives and the adaptation to multiple tasks. To tackle these challenges, we present two simple yet effective strategies. The first strategy, referred to as sequential learning, attempts to address how to optimize the diverse objectives, which guides the network to incrementally learn individual IR tasks in a sequential manner rather than mixing them together. The second strategy, i.e., prompt learning, attempts to address how to adapt to the different IR tasks, which assists the network to understand the specific task and improves the generalization ability. By evaluating on 19 test sets, we demonstrate that the sequential and prompt learning strategies can significantly enhance the MiO performance of commonly used CNN and Transformer backbones. Our experiments also reveal that the two strategies can supplement each other to learn better degradation representations and enhance the model robustness. It is expected that our proposed MiO IR formulation and strategies could facilitate the research on how to train IR models with higher generalization capabilities.