# Vision-Language Gradient Descent-driven All-in-One Deep Unfolding Networks

arXiv: https://arxiv.org/abs/2503.16930 · HTML: https://mlsanity.com/paper/2503.16930/

## Key facts

- Cited by 37 papers in the map (data through 14 Oct 2026).
- Builds on 34 of its 56 references that are in the map.
- First posted on arXiv 21 Mar 2025.

## Builds on (most cited first, 30 of 34)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation](https://mlsanity.com/paper/2201.12086/) (2022)
- [Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision](https://mlsanity.com/paper/2102.05918/) (2021)
- [Gaussian Error Linear Units (GELUs)](https://mlsanity.com/paper/1606.08415/) (2016)
- [SwinIR: Image Restoration Using Swin Transformer](https://mlsanity.com/paper/2108.10257/) (2021)
- [Restormer: Efficient Transformer for High-Resolution Image Restoration](https://mlsanity.com/paper/2111.09881/) (2021)
- [Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising](https://mlsanity.com/paper/1608.03981/) (2016)
- [Denoising Diffusion Restoration Models](https://mlsanity.com/paper/2201.11793/) (2022)
- [Multi-Stage Progressive Image Restoration](https://mlsanity.com/paper/2102.02808/) (2021)
- [FFDNet: Toward a Fast and Flexible Solution for CNN based Image Denoising](https://mlsanity.com/paper/1710.04026/) (2017)
- [MambaIR: A Simple Baseline for Image Restoration with State-Space Model](https://mlsanity.com/paper/2402.15648/) (2024)
- [DehazeNet: An End-to-End System for Single Image Haze Removal](https://mlsanity.com/paper/1601.07661/) (2016)
- [Multi-Scale Progressive Fusion Network for Single Image Deraining](https://mlsanity.com/paper/2003.10985/) (2020)
- [TransWeather: Transformer-based Restoration of Images Degraded by Adverse Weather Conditions](https://mlsanity.com/paper/2111.14813/) (2021)
- [Plug-and-Play Methods Provably Converge with Properly Trained Denoisers](https://mlsanity.com/paper/1905.05406/) (2019)
- [InstructIR: High-Quality Image Restoration Following Human Instructions](https://mlsanity.com/paper/2401.16468/) (2024)
- [Deep Generalized Unfolding Networks for Image Restoration](https://mlsanity.com/paper/2204.13348/) (2022)
- [Uncertainty Guided Multi-Scale Residual Learning-using a Cycle Spinning CNN for Single Image De-Raining](https://mlsanity.com/paper/1906.11129/) (2019)
- [Mask-guided Spectral-wise Transformer for Efficient Hyperspectral Image Reconstruction](https://mlsanity.com/paper/2111.07910/) (2021)
- [Controlling Vision-Language Models for Multi-Task Image Restoration](https://mlsanity.com/paper/2310.01018/) (2023)
- [ShadowDiffusion: When Degradation Prior Meets Diffusion Model for Shadow Removal](https://mlsanity.com/paper/2212.04711/) (2022)
- [Hierarchical Integration Diffusion Model for Realistic Image Deblurring](https://mlsanity.com/paper/2305.12966/) (2023)
- [Cascaded Deep Video Deblurring Using Temporal Sharpness Prior](https://mlsanity.com/paper/2004.02501/) (2020)
- [FD-GAN: Generative Adversarial Networks with Fusion-discriminator for Single Image Dehazing](https://mlsanity.com/paper/2001.06968/) (2020)
- [Coarse-to-Fine Sparse Transformer for Hyperspectral Image Reconstruction](https://mlsanity.com/paper/2203.04845/) (2022)
- [GridFormer: Residual Dense Transformer with Grid Structure for Image Restoration in Adverse Weather Conditions](https://mlsanity.com/paper/2305.17863/) (2023)
- [Perceive-IR: Learning to Perceive Degradation Better for All-in-One Image Restoration](https://mlsanity.com/paper/2408.15994/) (2024)
- [Online Video Deblurring via Dynamic Temporal Blending Network](https://mlsanity.com/paper/1704.03285/) (2017)
- [Neural Degradation Representation Learning for All-In-One Image Restoration](https://mlsanity.com/paper/2310.12848/) (2023)
- [DDS2M: Self-Supervised Denoising Diffusion Spatio-Spectral Model for Hyperspectral Image Restoration](https://mlsanity.com/paper/2303.06682/) (2023)

## Built on by (most cited first, 30 of 37)

- [ClearAIR: A Human-Visual-Perception-Inspired All-in-One Image Restoration](https://mlsanity.com/paper/2601.02763/) (2026)
- [Efficient Degradation-agnostic Image Restoration via Channel-Wise Functional Decomposition and Manifold Regularization](https://mlsanity.com/paper/2505.18679/) (2025)
- [Simulator and Experience Enhanced Diffusion Model for Comprehensive ECG Generation](https://mlsanity.com/paper/2511.09895/) (2025)
- [DSwinIR: Rethinking Window-based Attention for Image Restoration](https://mlsanity.com/paper/2504.04869/) (2025)
- [Decoupled Residual Denoising Diffusion Models for Unified and Data Efficient Image-to-Image Translation](https://mlsanity.com/paper/2606.01048/) (2026)
- [UniLDiff: Unlocking the Power of Diffusion Priors for All-in-One Image Restoration](https://mlsanity.com/paper/2507.23685/) (2025)
- [Universal Image Restoration Pre-training via Masked Degradation Classification](https://mlsanity.com/paper/2510.13282/) (2025)
- [TPGDiff: Hierarchical Triple-Prior Guided Diffusion for Image Restoration](https://mlsanity.com/paper/2601.20306/) (2026)
- [UDPNet: Unleashing Depth-based Priors for Robust Image Dehazing](https://mlsanity.com/paper/2601.06909/) (2026)
- [IRPO: Boosting Image Restoration via Post-training GRPO](https://mlsanity.com/paper/2512.00814/) (2025)
- [Task-Guided Prompting for Unified Remote Sensing Image Restoration](https://mlsanity.com/paper/2604.02742/) (2026)
- [Learning Adaptive Dynamical Features via Multi-τ Liquid-Mamba for All-in-one Image Restoration](https://mlsanity.com/paper/2606.22801/) (2026)
- [Clear Roads, Clear Vision: Advancements in Multi-Weather Restoration for Smart Transportation](https://mlsanity.com/paper/2510.09228/) (2025)
- [HSI-VAR: Rethinking Hyperspectral Restoration through Spatial-Spectral Visual Autoregression](https://mlsanity.com/paper/2602.00749/) (2026)
- [Deep LoRA-Unfolding Networks for Image Restoration](https://mlsanity.com/paper/2602.18697/) (2026)
- [Causal-AgentIR: Self-Evolving Causal Memory for Adaptive Image Restoration Agents](https://mlsanity.com/paper/2607.21125/) (2026)
- [Bend the Basics: Degradation-Aware Deformable Tokenization for All-in-One Image Restoration](https://mlsanity.com/paper/2608.06832/) (2026)
- [Diffusion Once and Done: Degradation-Aware LoRA for Efficient All-in-One Image Restoration](https://mlsanity.com/paper/2508.03373/) (2025)
- [Learning to Restore Multi-Degraded Images via Ingredient Decoupling and Task-Aware Path Adaptation](https://mlsanity.com/paper/2511.04920/) (2025)
- [Physically Interpretable Multi-Degradation Image Restoration via Deep Unfolding and Explainable Convolution](https://mlsanity.com/paper/2511.10166/) (2025)
- [Unleashing Degradation-Carrying Features in Symmetric U-Net: Simpler and Stronger Baselines for All-in-One Image Restoration](https://mlsanity.com/paper/2512.10581/) (2025)
- [From Physical Degradation Models to Task-Aware All-in-One Image Restoration](https://mlsanity.com/paper/2601.10192/) (2026)
- [Vision-Language Controlled Deep Unfolding for Joint Medical Image Restoration and Segmentation](https://mlsanity.com/paper/2601.23103/) (2026)
- [SLER-IR: Spherical Layer-wise Expert Routing for All-in-One Image Restoration](https://mlsanity.com/paper/2603.05940/) (2026)
- [Continuous Expert Assembly: Instance-Conditioned Low-Rank Residuals for All-in-One Image Restoration](https://mlsanity.com/paper/2605.06127/) (2026)
- [Leveraging Multimodal Large Language Models for All-in-One Image Restoration via a Mixture of Frequency Experts](https://mlsanity.com/paper/2605.11444/) (2026)
- [Multi-FRuGaL: Multimodal Flexible Redundancy-aware Decomposed Gated Learning for Cancer Diagnosis and Prognosis](https://mlsanity.com/paper/2606.06867/) (2026)
- [DVANet: Degradation-aware Visual-prior Alignment Network for Image Restoration](https://mlsanity.com/paper/2606.19097/) (2026)
- [CURE: Controllable Unified Image Restoration for Complex Degradations](https://mlsanity.com/paper/2607.03044/) (2026)
- [ExpoMotion: A Large-Scale Benchmark and A Householder Projection Network for Multi-Exposure Fusion](https://mlsanity.com/paper/2607.03110/) (2026)

## Abstract (arXiv)

Dynamic image degradations, including noise, blur and lighting inconsistencies, pose significant challenges in image restoration, often due to sensor limitations or adverse environmental conditions. Existing Deep Unfolding Networks (DUNs) offer stable restoration performance but require manual selection of degradation matrices for each degradation type, limiting their adaptability across diverse scenarios. To address this issue, we propose the Vision-Language-guided Unfolding Network (VLU-Net), a unified DUN framework for handling multiple degradation types simultaneously. VLU-Net leverages a Vision-Language Model (VLM) refined on degraded image-text pairs to align image features with degradation descriptions, selecting the appropriate transform for target degradation. By integrating an automatic VLM-based gradient estimation strategy into the Proximal Gradient Descent (PGD) algorithm, VLU-Net effectively tackles complex multi-degradation restoration tasks while maintaining interpretability. Furthermore, we design a hierarchical feature unfolding structure to enhance VLU-Net framework, efficiently synthesizing degradation patterns across various levels. VLU-Net is the first all-in-one DUN framework and outperforms current leading one-by-one and all-in-one end-to-end methods by 3.74 dB on the SOTS dehazing dataset and 1.70 dB on the Rain100L deraining dataset.