# Complexity Experts are Task-Discriminative Learners for Any Image Restoration

arXiv: https://arxiv.org/abs/2411.18466 · HTML: https://mlsanity.com/paper/2411.18466/

## Key facts

- Cited by 77 papers in the map (data through 14 Oct 2026).
- Placed in Vision mixture-of-experts, one of 12 directions in Mixture-of-experts.
- Builds on 50 of its 72 references that are in the map.
- First posted on arXiv 27 Nov 2024.

## Placed in

- Mixture-of-experts / [Vision mixture-of-experts](https://mlsanity.com/moe/vision-mixture-of-experts/)

## Builds on (most cited first, 30 of 50)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation](https://mlsanity.com/paper/1308.3432/) (2013)
- [SwinIR: Image Restoration Using Swin Transformer](https://mlsanity.com/paper/2108.10257/) (2021)
- [Restormer: Efficient Transformer for High-Resolution Image Restoration](https://mlsanity.com/paper/2111.09881/) (2021)
- [Uformer: A General U-Shaped Transformer for Image Restoration](https://mlsanity.com/paper/2106.03106/) (2021)
- [Multi-Stage Progressive Image Restoration](https://mlsanity.com/paper/2102.02808/) (2021)
- [Scaling Vision with Sparse Mixture of Experts](https://mlsanity.com/paper/2106.05974/) (2021)
- [Deep Retinex Decomposition for Low-Light Enhancement](https://mlsanity.com/paper/1808.04560/) (2018)
- [Deep Multi-scale Convolutional Neural Network for Dynamic Scene Deblurring](https://mlsanity.com/paper/1612.02177/) (2016)
- [Simple Baselines for Image Restoration](https://mlsanity.com/paper/2204.04676/) (2022)
- [Mixture-of-Experts with Expert Choice Routing](https://mlsanity.com/paper/2202.09368/) (2022)
- [Noise2Noise: Learning Image Restoration without Clean Data](https://mlsanity.com/paper/1803.04189/) (2018)
- [Benchmarking Single Image Dehazing and Beyond](https://mlsanity.com/paper/1712.04143/) (2017)
- [Learning Deep CNN Denoiser Prior for Image Restoration](https://mlsanity.com/paper/1704.03264/) (2017)
- [MambaIR: A Simple Baseline for Image Restoration with State-Space Model](https://mlsanity.com/paper/2402.15648/) (2024)
- [Learning Texture Transformer Network for Image Super-Resolution](https://mlsanity.com/paper/2006.04139/) (2020)
- [DehazeNet: An End-to-End System for Single Image Haze Removal](https://mlsanity.com/paper/1601.07661/) (2016)
- [Progressive Image Deraining Networks: A Better and Simpler Baseline](https://mlsanity.com/paper/1901.09221/) (2019)
- [Mixture-of-Depths: Dynamically allocating compute in transformer-based language models](https://mlsanity.com/paper/2404.02258/) (2024)
- [Multi-Scale Progressive Fusion Network for Single Image Deraining](https://mlsanity.com/paper/2003.10985/) (2020)
- [From Sparse to Soft Mixtures of Experts](https://mlsanity.com/paper/2308.00951/) (2023)
- [MemNet: A Persistent Memory Network for Image Restoration](https://mlsanity.com/paper/1708.02209/) (2017)
- [TransWeather: Transformer-based Restoration of Images Degraded by Adverse Weather Conditions](https://mlsanity.com/paper/2111.14813/) (2021)
- [Residual Non-local Attention Networks for Image Restoration](https://mlsanity.com/paper/1903.10082/) (2019)
- [Contrastive Learning for Compact Single Image Dehazing](https://mlsanity.com/paper/2104.09367/) (2021)
- [Restoring Vision in Adverse Weather Conditions with Patch-Based Denoising Diffusion Models](https://mlsanity.com/paper/2207.14626/) (2022)
- [Efficient Frequency Domain-based Transformers for High-Quality Image Deblurring](https://mlsanity.com/paper/2211.12250/) (2022)
- [InstructIR: High-Quality Image Restoration Following Human Instructions](https://mlsanity.com/paper/2401.16468/) (2024)
- [Gated Fusion Network for Single Image Dehazing](https://mlsanity.com/paper/1804.00213/) (2018)

## Built on by (most cited first, 30 of 77)

- [Mixture of Experts in Large Language Models](https://mlsanity.com/paper/2507.11181/) (2025)
- [Learning Continuous Wasserstein Barycenter Space for Generalized All-in-One Image Restoration](https://mlsanity.com/paper/2602.23169/) (2026)
- [UniRestorer: Universal Image Restoration via Adaptively Estimating Image Degradation at Proper Granularity](https://mlsanity.com/paper/2412.20157/) (2024)
- [Gradient as Conditions: Rethinking HOG for All-in-one Image Restoration](https://mlsanity.com/paper/2504.09377/) (2025)
- [FoundIR-v2: Optimizing Pre-Training Data Mixtures for Image Restoration Foundation Model](https://mlsanity.com/paper/2512.09282/) (2025)
- [RealRestorer: Towards Generalizable Real-World Image Restoration with Large-Scale Image Editing Models](https://mlsanity.com/paper/2603.25502/) (2026)
- [Efficient Degradation-agnostic Image Restoration via Channel-Wise Functional Decomposition and Manifold Regularization](https://mlsanity.com/paper/2505.18679/) (2025)
- [DSwinIR: Rethinking Window-based Attention for Image Restoration](https://mlsanity.com/paper/2504.04869/) (2025)
- [M2Restore: Mixture-of-Experts-based Mamba-CNN Fusion Framework for All-in-One Image Restoration](https://mlsanity.com/paper/2506.07814/) (2025)
- [EvoIR: Towards All-in-One Image Restoration via Evolutionary Frequency Modulation](https://mlsanity.com/paper/2512.05104/) (2025)
- [Learning Domain-Aware Task Prompt Representations for Multi-Domain All-in-One Image Restoration](https://mlsanity.com/paper/2603.01725/) (2026)
- [Low Light Image Enhancement Challenge at NTIRE 2026](https://mlsanity.com/paper/2604.17669/) (2026)
- [Beyond Degradation Redundancy: Contrastive Prompt Learning for All-in-One Image Restoration](https://mlsanity.com/paper/2504.09973/) (2025)
- [Bilevel Layer-Positioning LoRA for Real Image Dehazing](https://mlsanity.com/paper/2603.10872/) (2026)
- [ClusIR: Towards Cluster-Guided All-in-One Image Restoration](https://mlsanity.com/paper/2512.10948/) (2025)
- [Depth-Synergized Mamba Meets Memory Experts for All-Day Image Reflection Separation](https://mlsanity.com/paper/2601.00322/) (2026)
- [LoViF 2026 Challenge on Real-World All-in-One Image Restoration: Methods and Results](https://mlsanity.com/paper/2604.19445/) (2026)
- [Decoupled Residual Denoising Diffusion Models for Unified and Data Efficient Image-to-Image Translation](https://mlsanity.com/paper/2606.01048/) (2026)
- [Learning Where and What to Restore for Composite Image Restoration](https://mlsanity.com/paper/2503.17915/) (2025)
- [UniLDiff: Unlocking the Power of Diffusion Priors for All-in-One Image Restoration](https://mlsanity.com/paper/2507.23685/) (2025)
- [Universal Image Restoration Pre-training via Masked Degradation Classification](https://mlsanity.com/paper/2510.13282/) (2025)
- [FAPE-IR: Frequency-Aware Planning and Execution Framework for All-in-One Image Restoration](https://mlsanity.com/paper/2511.14099/) (2025)
- [DPMambaIR: All-in-One Image Restoration via Degradation-Aware Prompt State Space Model](https://mlsanity.com/paper/2504.17732/) (2025)
- [Latent Harmony: Synergistic Unified UHD Image Restoration via Latent Space Regularization and Controllable Refinement](https://mlsanity.com/paper/2510.07961/) (2025)
- [PaAgent: Portrait-Aware Image Restoration Agent via Subjective-Objective Reinforcement Learning](https://mlsanity.com/paper/2603.17055/) (2026)
- [A Unified Foundation Model for All-in-One Multi-Modal Remote Sensing Image Restoration and Fusion with Language Prompting](https://mlsanity.com/paper/2604.05629/) (2026)
- [EndoIR: Degradation-Agnostic All-in-One Endoscopic Image Restoration via Noise-Aware Routing Diffusion](https://mlsanity.com/paper/2511.05873/) (2025)
- [Task-Guided Prompting for Unified Remote Sensing Image Restoration](https://mlsanity.com/paper/2604.02742/) (2026)
- [EpiAgent: An Agent-Centric System for Ancient Inscription Restoration](https://mlsanity.com/paper/2604.09367/) (2026)
- [SplatWeaver: Learning to Allocate Gaussian Primitives for Generalizable Novel View Synthesis](https://mlsanity.com/paper/2605.07287/) (2026)

## Abstract (arXiv)

Recent advancements in all-in-one image restoration models have revolutionized the ability to address diverse degradations through a unified framework. However, parameters tied to specific tasks often remain inactive for other tasks, making mixture-of-experts (MoE) architectures a natural extension. Despite this, MoEs often show inconsistent behavior, with some experts unexpectedly generalizing across tasks while others struggle within their intended scope. This hinders leveraging MoEs' computational benefits by bypassing irrelevant experts during inference. We attribute this undesired behavior to the uniform and rigid architecture of traditional MoEs. To address this, we introduce ``complexity experts" -- flexible expert blocks with varying computational complexity and receptive fields. A key challenge is assigning tasks to each expert, as degradation complexity is unknown in advance. Thus, we execute tasks with a simple bias toward lower complexity. To our surprise, this preference effectively drives task-specific allocation, assigning tasks to experts with the appropriate complexity. Extensive experiments validate our approach, demonstrating the ability to bypass irrelevant experts during inference while maintaining superior performance. The proposed MoCE-IR model outperforms state-of-the-art methods, affirming its efficiency and practical applicability. The source code and models are publicly available at hrefhttps://eduardzamfir.github.io/moceir/texttteduardzamfir.github.io/MoCE-IR/