mlsanityData to 14 Oct 2026

RestoreAgent: Autonomous Image Restoration Agent via Multimodal Large Language Models

arXiv 2407.18035 ↗First posted 25 Jul 2024cs.CV cs.AI cs.CLcited by 43

On the map: Multimodal LLMs / vision-language models → Image editing and quality assessment

also in 1 more
Abstract (arXiv)

Natural images captured by mobile devices often suffer from multiple types of degradation, such as noise, blur, and low light. Traditional image restoration methods require manual selection of specific tasks, algorithms, and execution sequences, which is time-consuming and may yield suboptimal results. All-in-one models, though capable of handling multiple tasks, typically support only a limited range and often produce overly smooth, low-fidelity outcomes due to their broad data distribution fitting. To address these challenges, we first define a new pipeline for restoring images with multiple degradations, and then introduce RestoreAgent, an intelligent image restoration system leveraging multimodal large language models. RestoreAgent autonomously assesses the type and extent of degradation in input images and performs restoration through (1) determining the appropriate restoration tasks, (2) optimizing the task sequence, (3) selecting the most suitable models, and (4) executing the restoration. Experimental results demonstrate the superior performance of RestoreAgent in handling complex degradation, surpassing human experts. Furthermore, the system modular design facilitates the fast integration of new tasks and models, enhancing its flexibility and scalability for various applications.

Key facts. Cited by 43 papers in the map (data through 14 Oct 2026). Placed in Image editing and quality assessment, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 51 of its 61 references that are in the map. First posted on arXiv 25 Jul 2024.

Builds on

51 of 61 references.

Closest ancestors · 1

Same direction, newest first

  1. 2024LLMRA: Multi-modal Large Language Model based Restoration Assistant21 Jan

Other references · 50

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,380
  3. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 14,132
  4. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,177
  5. 2023PaLM-E: An Embodied Multimodal Language Modelcited by 2,071
  6. 2021Restormer: Efficient Transformer for High-Resolution Image Restorationcited by 1,627
  7. 2016Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoisingcited by 1,358
  8. 2023Kosmos-2: Grounding Multimodal Large Language Models to the Worldcited by 1,089
  9. 2021Real-ESRGAN: Training Real-World Blind Super-Resolution with Pure Synthetic Datacited by 970
  10. 2020Image Quality Assessment: Unifying Structure and Texture Similaritycited by 811
  11. 2023MiniGPT-v2: large language model as a unified interface for vision-language multi-task learningcited by 625
  12. 2018Noise2Noise: Learning Image Restoration without Clean Datacited by 533
38 more
  1. 2023Exploiting Diffusion Prior for Real-World Image Super-Resolutioncited by 489
  2. 2023GLaMM: Pixel Grounding Large Multimodal Modelcited by 455
  3. 2017DeblurGAN: Blind Motion Deblurring Using Conditional Adversarial Networkscited by 367
  4. 2021Rethinking Coarse-to-Fine Approach in Single Image Deblurringcited by 326
  5. 2023SeeSR: Towards Semantics-Aware Real-World Image Super-Resolutioncited by 315
  6. 2019FFA-Net: Feature Fusion Attention Network for Single Image Dehazingcited by 307
  7. 2024Scaling Up to Excellence: Practicing Model Scaling for Photo-Realistic Image Restoration In the Wildcited by 286
  8. 2023DiffBIR: Towards Blind Image Restoration with Generative Diffusion Priorcited by 257
  9. 2021Deblurring via Stochastic Refinementcited by 249
  10. 2019Residual Non-local Attention Networks for Image Restorationcited by 236
  11. 2023Retinexformer: One-stage Retinex-based Transformer for Low-light Image Enhancementcited by 231
  12. 2021Contrastive Learning for Compact Single Image Dehazingcited by 219
  13. 2023Learning A Sparse Transformer Network for Effective Image Derainingcited by 205
  14. 2020PIPAL: a Large-Scale Image Quality Assessment Dataset for Perceptual Image Restorationcited by 178
  15. 2019High-Quality Self-Supervised Deep Image Denoisingcited by 161
  16. 2024InstructIR: High-Quality Image Restoration Following Human Instructionscited by 140
  17. 2023PromptIR: Prompting for All-in-One Blind Image Restorationcited by 114
  18. 2024Revisiting Zeroth-Order Optimization for Memory-Efficient LLM Fine-Tuning: A Benchmarkcited by 114
  19. 2023AutoDIR: Automatic All-in-One Image Restoration with Latent Diffusioncited by 100
  20. 2022Blind2Unblind: Self-Supervised Image Denoising with Visible Blind Spotscited by 84
  21. 2022Multiscale Structure Guided Diffusion for Image Deblurringcited by 77
  22. 2021Towards Flexible Blind JPEG Artifacts Removalcited by 76
  23. 2020Unpaired Learning of Deep Image Denoisingcited by 74
  24. 2023RIDCP: Revitalizing Real Image Dehazing via High-Quality Codebook Priorscited by 73
  25. 2023CoSeR: Bridging Image and Language for Cognitive Super-Resolutioncited by 66
  26. 2023Masked Image Training for Generalizable Deep Image Denoisingcited by 57
  27. 2022TAPE: Task-Agnostic Prior Embedding for Image Restorationcited by 55
  28. 2023Multimodal Prompt Perceiver: Empower Adaptiveness, Generalizability and Fidelity for All-in-One Image Restorationcited by 50
  29. 2024Towards Effective Multiple-in-One Image Restoration: A Sequential and Prompt Learning Strategycited by 50
  30. 2018Crafting a Toolchain for Image Restoration by Deep Reinforcement Learningcited by 47
  31. 2023Sparse Sampling Transformer with Uncertainty-Driven Ranking for Unified Removal of Raindrops and Rain Streakscited by 45
  32. 2023Prompt-In-Prompt Learning for Universal Image Restorationcited by 43
  33. 2021IDR: Self-Supervised Image Denoising via Iterative Data Refinementcited by 36
  34. 2020Image Quality Assessment for Perceptual Image Restoration: A New Dataset, Benchmark and Metriccited by 26
  35. 2024Low-Res Leads the Way: Improving Generalization for Super-Resolution by Self-Supervised Learningcited by 19
  36. 2023Score Priors Guided Deep Variational Inference for Unsupervised Real-World Single Image Denoisingcited by 15
  37. 2022SnowFormer: Context Interaction Transformer with Scale-awareness for Single Image Desnowingcited by 13
  38. 2023Clarity ChatGPT: An Interactive and Adaptive Processing System for Image Restoration and Enhancementcited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 6; 2025: 18; 2026: 19.

43 papers.

Sort:

By area:

  1. 2024A Survey on All-in-One Image Restoration: Taxonomy, Evaluation and Future Trendscited by 60
  2. 2024An Intelligent Agentic System for Complex Image Restoration Problemscited by 43
  3. 2024FoundIR: Unleashing Million-scale Training Data to Advance Foundation Models for Image Restorationcited by 39
  4. 2025POSTA: A Go-to Framework for Customized Artistic Poster Generationcited by 36
  5. 2025JarvisIR: Elevating Autonomous Driving Perception with Intelligent Image Restorationcited by 34
  6. 2024Unveiling Hidden Details: A RAW Data-Enhanced Paradigm for Real-World Super-Resolutioncited by 26
  7. 2025Multi-Agent Image Restorationcited by 21
  8. 2025Hybrid Agents for Image Restorationcited by 17
  9. 2024UniRestorer: Universal Image Restoration via Adaptively Estimating Image Degradation at Proper Granularitycited by 14
  10. 2025MoA-VR: A Mixture-of-Agents System Towards All-in-One Video Restorationcited by 12
  11. 2025Follow-Your-Instruction: A Comprehensive MLLM Agent for World Data Synthesiscited by 10
  12. 2025When Continue Learning Meets Multimodal Large Language Model: A Surveycited by 9
31 more
  1. 2025Towards Robust Evaluation of STEM Education: Leveraging MLLMs in Project-Based Learningcited by 7
  2. 2025WeatherPrompt: Multi-modality Representation Learning for All-Weather Drone Visual Geo-Localizationcited by 7
  3. 2026Agentic Retoucher for Text-To-Image Generationcited by 7
  4. 2025Restore-R1: Efficient Image Restoration Agents via Reinforcement Learning with Multimodal LLM Perceptual Feedbackcited by 6
  5. 2025UniRes: Universal Image Restoration for Complex Degradationscited by 5
  6. 2025Multi-level Value Alignment in Agentic AI Systems: Survey and Perspectivescited by 5
  7. 2026TIR-Agent: Training an Explorative and Efficient Agent for Image Restorationcited by 5
  8. 2024Chain-of-Restoration: Multi-Task Image Restoration Models are Zero-Shot Step-by-Step Universal Image Restorerscited by 4
  9. 2025Exploring Scalable Unified Modeling for General Low-Level Visioncited by 4
  10. 2025PhyDAE: Physics-Guided Degradation-Adaptive Experts for All-in-One Remote Sensing Image Restorationcited by 4
  11. 2026RetouchIQ: MLLM Agents for Instruction-Based Image Retouching with Generalist Rewardcited by 4
  12. 2026OPERA: An Agent for Image Restoration with End-to-End Joint Planning-Execution Optimizationcited by 4
  13. 2026Self-Evolving Agentic Image Restoration via Deliberate Planning and Intuitive Executioncited by 4
  14. 2025Position: Agentic Systems Constitute a Key Component of Next-Generation Intelligent Image Processingcited by 3
  15. 2026PaAgent: Portrait-Aware Image Restoration Agent via Subjective-Objective Reinforcement Learningcited by 3
  16. 2026Restore, Assess, Repeat: A Unified Framework for Iterative Image Restorationcited by 3
  17. 2025Real-World Adverse Weather Image Restoration via Dual-Level Reinforcement Learning with High-Quality Cold Startcited by 2
  18. 2026EpiAgent: An Agent-Centric System for Ancient Inscription Restorationcited by 2
  19. 2026DiTTo: Scalable Order-aware All-in-One Image Restoration Agentcited by 2
  20. 2025OmniQuality-R: Advancing Reward Models Through All-Encompassing Quality Assessmentcited by 1
  21. 2026Derain-Agent: A Plug-and-Play Agent Framework for Rainy Image Restorationcited by 1
  22. 2026Causal-AgentIR: Self-Evolving Causal Memory for Adaptive Image Restoration Agentscited by 1
  23. 2025Unleashing Degradation-Carrying Features in Symmetric U-Net: Simpler and Stronger Baselines for All-in-One Image Restorationcited by 0
  24. 2026VQ-Jarvis: Retrieval-Augmented Video Restoration Agent with Sharp Vision and Fast Thoughtcited by 0
  25. 2026Leveraging Multimodal Large Language Models for All-in-One Image Restoration via a Mixture of Frequency Expertscited by 0
  26. 2026EvoIR-Agent: Self-Evolving Image Restoration Agentic System via Experience-Driven Learningcited by 0
  27. 2026VLM- and LLM-Driven Multi-Agent System for PET Image Denoisingcited by 0
  28. 2026FRAMEWORKERS: A Dynamic Multi-Agent Framework for AI-Generated Video Productioncited by 0
  29. 2026Geometric Similarity in VLM Low-Level Vision Representationscited by 0
  30. 2026EORestore-Agent: Fidelity-Guided Agentic Restoration of Remote Sensing Images with Composite Degradationscited by 0
  31. 2026Look Before You Restore: Learning to Predict Before Acting for Agentic Image Restorationcited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2026HarnessIR: Harnessing Multimodal Foundation Models for Universal Real-World Image Restoration
  2. 2026SkillIR: Evolving Scene-Aware Skills for Agentic Image Restoration
  3. 2024Restore Anything Model via Efficient Degradation Adaptation
  4. 2026RIRF: Reasoning Image Restoration Framework
  5. 2024LossAgent: Towards Any Optimization Objectives for Image Processing with LLM Agents
  6. 2026ClearAIR: A Human-Visual-Perception-Inspired All-in-One Image Restoration
  7. 2025An Efficient and Mixed Heterogeneous Model for Image Restoration
  8. 2026UnSCAR: Universal, Scalable, Controllable, and Adaptable Image Restoration