# Learning Where and What to Restore for Composite Image Restoration

arXiv: https://arxiv.org/abs/2503.17915 · HTML: https://mlsanity.com/paper/2503.17915/

## Key facts

- Cited by 4 papers in the map (data through 14 Oct 2026).
- Builds on 37 of its 55 references that are in the map.
- First posted on arXiv 23 Mar 2025.

## Builds on (most cited first, 30 of 37)

- [Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer](https://mlsanity.com/paper/1701.06538/) (2017)
- [Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity](https://mlsanity.com/paper/2101.03961/) (2021)
- [Categorical Reparameterization with Gumbel-Softmax](https://mlsanity.com/paper/1611.01144/) (2016)
- [Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation](https://mlsanity.com/paper/1308.3432/) (2013)
- [SwinIR: Image Restoration Using Swin Transformer](https://mlsanity.com/paper/2108.10257/) (2021)
- [Restormer: Efficient Transformer for High-Resolution Image Restoration](https://mlsanity.com/paper/2111.09881/) (2021)
- [Swin Transformer V2: Scaling Up Capacity and Resolution](https://mlsanity.com/paper/2111.09883/) (2021)
- [Pre-Trained Image Processing Transformer](https://mlsanity.com/paper/2012.00364/) (2020)
- [Uformer: A General U-Shaped Transformer for Image Restoration](https://mlsanity.com/paper/2106.03106/) (2021)
- [Multi-Stage Progressive Image Restoration](https://mlsanity.com/paper/2102.02808/) (2021)
- [Deep Multi-scale Convolutional Neural Network for Dynamic Scene Deblurring](https://mlsanity.com/paper/1612.02177/) (2016)
- [Deep Retinex Decomposition for Low-Light Enhancement](https://mlsanity.com/paper/1808.04560/) (2018)
- [Prevalence of Neural Collapse during the terminal phase of deep learning training](https://mlsanity.com/paper/2008.08186/) (2020)
- [Simple Baselines for Image Restoration](https://mlsanity.com/paper/2204.04676/) (2022)
- [Benchmarking Single Image Dehazing and Beyond](https://mlsanity.com/paper/1712.04143/) (2017)
- [Deep Joint Rain Detection and Removal from a Single Image](https://mlsanity.com/paper/1609.07769/) (2016)
- [MambaIR: A Simple Baseline for Image Restoration with State-Space Model](https://mlsanity.com/paper/2402.15648/) (2024)
- [Learning Enriched Features for Real Image Restoration and Enhancement](https://mlsanity.com/paper/2003.06792/) (2020)
- [TransWeather: Transformer-based Restoration of Images Degraded by Adverse Weather Conditions](https://mlsanity.com/paper/2111.14813/) (2021)
- [A Geometric Analysis of Neural Collapse with Unconstrained Features](https://mlsanity.com/paper/2105.02375/) (2021)
- [Restoring Vision in Adverse Weather Conditions with Patch-Based Denoising Diffusion Models](https://mlsanity.com/paper/2207.14626/) (2022)
- [CycleISP: Real Image Restoration via Improved Data Synthesis](https://mlsanity.com/paper/2003.07761/) (2020)
- [InstructIR: High-Quality Image Restoration Following Human Instructions](https://mlsanity.com/paper/2401.16468/) (2024)
- [Deep Generalized Unfolding Networks for Image Restoration](https://mlsanity.com/paper/2204.13348/) (2022)
- [AdaIR: Adaptive All-in-One Image Restoration via Frequency Mining and Modulation](https://mlsanity.com/paper/2403.14614/) (2024)
- [OneRestore: A Universal Restoration Framework for Composite Degradation](https://mlsanity.com/paper/2407.04621/) (2024)
- [Complexity Experts are Task-Discriminative Learners for Any Image Restoration](https://mlsanity.com/paper/2411.18466/) (2024)
- [GridFormer: Residual Dense Transformer with Grid Structure for Image Restoration in Adverse Weather Conditions](https://mlsanity.com/paper/2305.17863/) (2023)
- [Multimodal Prompt Perceiver: Empower Adaptiveness, Generalizability and Fidelity for All-in-One Image Restoration](https://mlsanity.com/paper/2312.02918/) (2023)
- [Path-Restore: Learning Network Path Selection for Image Restoration](https://mlsanity.com/paper/1904.10343/) (2019)

## Built on by (most cited first, 4 of 4)

- [ClusIR: Towards Cluster-Guided All-in-One Image Restoration](https://mlsanity.com/paper/2512.10948/) (2025)
- [Clear Roads, Clear Vision: Advancements in Multi-Weather Restoration for Smart Transportation](https://mlsanity.com/paper/2510.09228/) (2025)
- [Pure-Pass: Fine-Grained, Adaptive Masking for Dynamic Token-Mixing Routing in Lightweight Image Super-Resolution](https://mlsanity.com/paper/2510.01997/) (2025)
- [Beyond Uniform Restoration: Empowering All-in-One Restoration with Pixel-Level Multimodal Guidance](https://mlsanity.com/paper/2608.09482/) (2026)

## Abstract (arXiv)

Real-world degraded images often contain multiple co-occurring degradation types, making composite image restoration fundamentally different from the single-degradation setting assumed by most existing all-in-one methods. These methods typically apply uniform spatial computation and single-label task conditioning, limiting both spatial adaptivity and explicit modeling of degradation mixtures. We propose CART (Composite-Adaptive Routing and Task Conditioning), a unified framework that jointly decides where to spend computation and what degradation cues should guide restoration: a spatial mixer routes patches to graded-capacity experts by local restoration difficulty, and a channel mixer routes each image through degradation-specific experts, conditioned on the global task feature with multi-hot classification supervision. Empirically, CART achieves state-of-the-art performance for composite degradation restoration on CDD-11 and delivers the best results to date on the conventional 3-task and 5-task all-in-one benchmarks.