mlsanityData to 14 Oct 2026

Leveraging Multimodal Large Language Models for All-in-One Image Restoration via a Mixture of Frequency Experts

arXiv 2605.11444 ↗First posted 12 May 2026cs.CVcited by 0

On the map: Multimodal LLMs / vision-language models → Image editing and quality assessment

Abstract (arXiv)

All-in-one image restoration seeks to recover clean images from inputs affected by diverse and unknown degradations using a unified framework. Recent methods have shown strong performance by identifying degradation characteristics to guide the restoration process. However, many of them treat degradations as discrete categories, which limits their ability to model the continuous relational structure that arises in composite degradations. To address this issue, we propose a multimodal large language model (MLLM)-guided image restoration framework that exploits multimodal embeddings as guidance for low-level restoration. Specifically, MLLM-derived features are injected into an encoder-decoder architecture through an MLLM-guided fusion block (MGFB) to enhance degradation-aware representations. In addition, we incorporate a mixture-of-frequency-experts (MoFE) module that adaptively combines frequency experts using MLLM-guided contextual cues. To further improve expert routing, we design an MLLM-guided router with a relational alignment loss that encourages routing patterns consistent with the embedding-space relationships of degraded inputs. Extensive experiments on multiple benchmarks show that the proposed method achieves strong performance across diverse restoration settings and establishes a new state of the art on the challenging CDD11 dataset, outperforming previous methods by up to 1.35 dB.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in Image editing and quality assessment, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 62 of its 79 references that are in the map. First posted on arXiv 12 May 2026.

Builds on

62 of 79 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024RestoreAgent: Autonomous Image Restoration Agent via Multimodal Large Language Models25 Jul
  2. 2023Controlling Vision-Language Models for Multi-Task Image Restoration2 Oct

Other references · 60

  1. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  2. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  3. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,863
  4. 2023GPT-4 Technical Reportcited by 18,396
  5. 2019Sentence-BERT: Sentence Embeddings using Siamese BERT-Networkscited by 9,170
  6. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
  7. 2025Qwen2.5-VL Technical Reportcited by 5,587
  8. 2023Improved Baselines with Visual Instruction Tuningcited by 4,418
  9. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,322
  10. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
  11. 2022LAION-5B: An open large-scale dataset for training next generation image-text modelscited by 4,152
  12. 2022DreamFusion: Text-to-3D using 2D Diffusioncited by 3,025
38 more
  1. 2023InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Taskscited by 2,565
  2. 2021SwinIR: Image Restoration Using Swin Transformercited by 1,887
  3. 2021Restormer: Efficient Transformer for High-Resolution Image Restorationcited by 1,627
  4. 2023T2I-Adapter: Learning Adapters to Dig out More Controllable Ability for Text-to-Image Diffusion Modelscited by 1,351
  5. 2021Uformer: A General U-Shaped Transformer for Image Restorationcited by 866
  6. 2021Multi-Stage Progressive Image Restorationcited by 816
  7. 2016Deep Multi-scale Convolutional Neural Network for Dynamic Scene Deblurringcited by 727
  8. 2018Deep Retinex Decomposition for Low-Light Enhancementcited by 723
  9. 2022Simple Baselines for Image Restorationcited by 708
  10. 2021RegionCLIP: Region-based Language-Image Pretrainingcited by 616
  11. 2021DenseCLIP: Language-Guided Dense Prediction with Context-Aware Promptingcited by 532
  12. 2017Benchmarking Single Image Dehazing and Beyondcited by 447
  13. 2017Learning Deep CNN Denoiser Prior for Image Restorationcited by 408
  14. 2021CRIS: CLIP-Driven Referring Image Segmentationcited by 353
  15. 2020Learning Texture Transformer Network for Image Super-Resolutioncited by 347
  16. 2022How Do Vision Transformers Work?cited by 307
  17. 2018Toward Convolutional Blind Denoising of Real Photographscited by 274
  18. 2019Progressive Image Deraining Networks: A Better and Simpler Baselinecited by 272
  19. 2020Multi-Scale Progressive Fusion Network for Single Image Derainingcited by 260
  20. 2021TransWeather: Transformer-based Restoration of Images Degraded by Adverse Weather Conditionscited by 244
  21. 2019Residual Non-local Attention Networks for Image Restorationcited by 236
  22. 2022Restoring Vision in Adverse Weather Conditions with Patch-Based Denoising Diffusion Modelscited by 215
  23. 2020CycleISP: Real Image Restoration via Improved Data Synthesiscited by 213
  24. 2020Deblurring by Realistic Blurringcited by 188
  25. 2018Densely Connected Pyramid Dehazing Networkcited by 145
  26. 2024InstructIR: High-Quality Image Restoration Following Human Instructionscited by 140
  27. 2022Anti-Oversmoothing in Deep Vision Transformers via the Fourier Domain Analysis: From Theory to Practicecited by 136
  28. 2018Gated Fusion Network for Single Image Dehazingcited by 133
  29. 2024AdaIR: Adaptive All-in-One Image Restoration via Frequency Mining and Modulationcited by 111
  30. 2023AutoDIR: Automatic All-in-One Image Restoration with Latent Diffusioncited by 100
  31. 2024OneRestore: A Universal Restoration Framework for Composite Degradationcited by 92
  32. 2024Complexity Experts are Task-Discriminative Learners for Any Image Restorationcited by 77
  33. 2024FastVLM: Efficient Vision Encoding for Vision Language Modelscited by 77
  34. 2019Dual Residual Networks Leveraging the Potential of Paired Operations for Image Restorationcited by 73
  35. 2024DreamClear: High-Capacity Real-World Image Restoration with Privacy-Safe Dataset Curationcited by 65
  36. 2023GridFormer: Residual Dense Transformer with Grid Structure for Image Restoration in Adverse Weather Conditionscited by 62
  37. 2023Multimodal Prompt Perceiver: Empower Adaptiveness, Generalizability and Fidelity for All-in-One Image Restorationcited by 50
  38. 2023Neural Degradation Representation Learning for All-In-One Image Restorationcited by 46

50 most cited of 60 All 60 (JSON)

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2024LLMRA: Multi-modal Large Language Model based Restoration Assistant
  2. 2025ClusIR: Towards Cluster-Guided All-in-One Image Restoration
  3. 2026M2IR: Proactive All-in-One Image Restoration via Mamba-style Modulation and Mixture-of-Experts
  4. 2025Restore-R1: Efficient Image Restoration Agents via Reinforcement Learning with Multimodal LLM Perceptual Feedback
  5. 2026UnSCAR: Universal, Scalable, Controllable, and Adaptable Image Restoration
  6. 2026Continuous Expert Assembly: Instance-Conditioned Low-Rank Residuals for All-in-One Image Restoration
  7. 2026DPC-Net: Dual-Prior Collaborative Network for All-in-One Image Restoration
  8. 2024Chain-of-Restoration: Multi-Task Image Restoration Models are Zero-Shot Step-by-Step Universal Image Restorers