# Language-driven All-in-one Adverse Weather Removal

arXiv: https://arxiv.org/abs/2312.01381 · HTML: https://mlsanity.com/paper/2312.01381/

## Key facts

- Cited by 36 papers in the map (data through 14 Oct 2026).
- Placed in Vision mixture-of-experts, one of 12 directions in Mixture-of-experts.
- Builds on 36 of its 59 references that are in the map.
- First posted on arXiv 3 Dec 2023.

## Placed in

- Mixture-of-experts / [Vision mixture-of-experts](https://mlsanity.com/moe/vision-mixture-of-experts/)

## Builds on (most cited first, 30 of 36)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Visual Instruction Tuning](https://mlsanity.com/paper/2304.08485/) (2023)
- [BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models](https://mlsanity.com/paper/2301.12597/) (2023)
- [BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation](https://mlsanity.com/paper/2201.12086/) (2022)
- [Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer](https://mlsanity.com/paper/1701.06538/) (2017)
- [MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Models](https://mlsanity.com/paper/2304.10592/) (2023)
- [Restormer: Efficient Transformer for High-Resolution Image Restoration](https://mlsanity.com/paper/2111.09881/) (2021)
- [Planning-oriented Autonomous Driving](https://mlsanity.com/paper/2212.10156/) (2022)
- [GLaM: Efficient Scaling of Language Models with Mixture-of-Experts](https://mlsanity.com/paper/2112.06905/) (2021)
- [Pre-Trained Image Processing Transformer](https://mlsanity.com/paper/2012.00364/) (2020)
- [Uformer: A General U-Shaped Transformer for Image Restoration](https://mlsanity.com/paper/2106.03106/) (2021)
- [Multi-Stage Progressive Image Restoration](https://mlsanity.com/paper/2102.02808/) (2021)
- [LISA: Reasoning Segmentation via Large Language Model](https://mlsanity.com/paper/2308.00692/) (2023)
- [Simple Baselines for Image Restoration](https://mlsanity.com/paper/2204.04676/) (2022)
- [FFA-Net: Feature Fusion Attention Network for Single Image Dehazing](https://mlsanity.com/paper/1911.07559/) (2019)
- [Vision Transformers for Single Image Dehazing](https://mlsanity.com/paper/2204.03883/) (2022)
- [TransWeather: Transformer-based Restoration of Images Degraded by Adverse Weather Conditions](https://mlsanity.com/paper/2111.14813/) (2021)
- [HINet: Half Instance Normalization Network for Image Restoration](https://mlsanity.com/paper/2105.06086/) (2021)
- [Restoring Vision in Adverse Weather Conditions with Patch-Based Denoising Diffusion Models](https://mlsanity.com/paper/2207.14626/) (2022)
- [Learning A Sparse Transformer Network for Effective Image Deraining](https://mlsanity.com/paper/2303.11950/) (2023)
- [Attentive Generative Adversarial Network for Raindrop Removal from a Single Image](https://mlsanity.com/paper/1711.10098/) (2017)
- [DesnowNet: Context-Aware Deep Network for Snow Removal](https://mlsanity.com/paper/1708.04512/) (2017)
- [Heavy Rain Image Restoration: Integrating Physics Model and Conditional Adversarial Learning](https://mlsanity.com/paper/1904.05050/) (2019)
- [Efficient and Explicit Modelling of Image Hierarchies for Image Restoration](https://mlsanity.com/paper/2303.00748/) (2023)
- [Bringing a Blurry Frame Alive at High Frame-Rate with an Event Camera](https://mlsanity.com/paper/1811.10180/) (2018)
- [Iterative Prompt Learning for Unsupervised Backlit Image Enhancement](https://mlsanity.com/paper/2303.17569/) (2023)
- [Neural Map Prior for Autonomous Driving](https://mlsanity.com/paper/2304.08481/) (2023)
- [Event Camera Data Pre-training](https://mlsanity.com/paper/2301.01928/) (2023)
- [MoEBERT: from BERT to Mixture-of-Experts via Importance-Guided Adaptation](https://mlsanity.com/paper/2204.07675/) (2022)
- [DRT: A Lightweight Single Image Deraining Recursive Transformer](https://mlsanity.com/paper/2204.11385/) (2022)

## Built on by (most cited first, 30 of 36)

- [AdaIR: Adaptive All-in-One Image Restoration via Frequency Mining and Modulation](https://mlsanity.com/paper/2403.14614/) (2024)
- [Complexity Experts are Task-Discriminative Learners for Any Image Restoration](https://mlsanity.com/paper/2411.18466/) (2024)
- [A Survey on All-in-One Image Restoration: Taxonomy, Evaluation and Future Trends](https://mlsanity.com/paper/2410.15067/) (2024)
- [Multi-Expert Adaptive Selection: Task-Balancing for All-in-One Image Restoration](https://mlsanity.com/paper/2407.19139/) (2024)
- [LoRA-IR: Taming Low-Rank Experts for Efficient All-in-One Image Restoration](https://mlsanity.com/paper/2410.15385/) (2024)
- [Diff-Restorer: Unleashing Visual Prompts for Diffusion-based Universal Image Restoration](https://mlsanity.com/paper/2407.03636/) (2024)
- [DGSolver: Diffusion Generalist Solver with Universal Posterior Sampling for Image Restoration](https://mlsanity.com/paper/2504.21487/) (2025)
- [ClusIR: Towards Cluster-Guided All-in-One Image Restoration](https://mlsanity.com/paper/2512.10948/) (2025)
- [Prompt to Restore, Restore to Prompt: Cyclic Prompting for Universal Adverse Weather Removal](https://mlsanity.com/paper/2503.09013/) (2025)
- [Adaptive Blind All-in-One Image Restoration](https://mlsanity.com/paper/2411.18412/) (2024)
- [LINK: Adaptive Modality Interaction for Audio-Visual Video Parsing](https://mlsanity.com/paper/2412.20872/) (2024)
- [Elucidating and Endowing the Diffusion Training Paradigm for General Image Restoration](https://mlsanity.com/paper/2506.21722/) (2025)
- [MUG: Pseudo Labeling Augmented Audio-Visual Mamba Network for Audio-Visual Video Parsing](https://mlsanity.com/paper/2507.01384/) (2025)
- [Grounding Degradations in Natural Language for All-In-One Video Restoration](https://mlsanity.com/paper/2507.14851/) (2025)
- [Robust Adverse Weather Removal via Spectral-based Spatial Grouping](https://mlsanity.com/paper/2507.22498/) (2025)
- [AWM-Fuse: Multi-Modality Image Fusion for Adverse Weather via Global and Local Text Perception](https://mlsanity.com/paper/2508.16881/) (2025)
- [LMHaze: Intensity-aware Image Dehazing with a Large-scale Multi-intensity Real Haze Dataset](https://mlsanity.com/paper/2410.16095/) (2024)
- [Clear Nights Ahead: Towards Multi-Weather Nighttime Image Restoration](https://mlsanity.com/paper/2505.16479/) (2025)
- [Real-World Adverse Weather Image Restoration via Dual-Level Reinforcement Learning with High-Quality Cold Start](https://mlsanity.com/paper/2511.05095/) (2025)
- [All-in-One Video Restoration under Smoothly Evolving Unknown Weather Degradations](https://mlsanity.com/paper/2601.00533/) (2026)
- [Accurate Forgetting for All-in-One Image Restoration Model](https://mlsanity.com/paper/2409.00685/) (2024)
- [A Preliminary Study on GPT-Image Generation Model for Image Restoration](https://mlsanity.com/paper/2505.05621/) (2025)
- [TAP: Parameter-efficient Task-Aware Prompting for Adverse Weather Removal](https://mlsanity.com/paper/2508.07878/) (2025)
- [Clear Roads, Clear Vision: Advancements in Multi-Weather Restoration for Smart Transportation](https://mlsanity.com/paper/2510.09228/) (2025)
- [Degradation-Aware Prompt Learning with Cross-Modal Compensation for Adverse Weather Removal](https://mlsanity.com/paper/2608.06939/) (2026)
- [Learning Dual Transformers for All-In-One Image Restoration from a Frequency Perspective](https://mlsanity.com/paper/2407.01636/) (2024)
- [MdaIF: Robust One-Stop Multi-Degradation-Aware Image Fusion with Language-Driven Semantics](https://mlsanity.com/paper/2511.12525/) (2025)
- [VLM-Augmented Degradation Modeling for Image Restoration Under Adverse Weather Conditions](https://mlsanity.com/paper/2511.16998/) (2025)
- [Unleashing Degradation-Carrying Features in Symmetric U-Net: Simpler and Stronger Baselines for All-in-One Image Restoration](https://mlsanity.com/paper/2512.10581/) (2025)
- [DELNet: Continuous All-in-One Weather Removal via Dynamic Expert Library](https://mlsanity.com/paper/2601.22573/) (2026)

## Abstract (arXiv)

All-in-one (AiO) frameworks restore various adverse weather degradations with a single set of networks jointly. To handle various weather conditions, an AiO framework is expected to adaptively learn weather-specific knowledge for different degradations and shared knowledge for common patterns. However, existing methods: 1) rely on extra supervision signals, which are usually unknown in real-world applications; 2) employ fixed network structures, which restrict the diversity of weather-specific knowledge. In this paper, we propose a Language-driven Restoration framework (LDR) to alleviate the aforementioned issues. First, we leverage the power of pre-trained vision-language (PVL) models to enrich the diversity of weather-specific knowledge by reasoning about the occurrence, type, and severity of degradation, generating description-based degradation priors. Then, with the guidance of degradation prior, we sparsely select restoration experts from a candidate list dynamically based on a Mixture-of-Experts (MoE) structure. This enables us to adaptively learn the weather-specific and shared knowledge to handle various weather conditions (e.g., unknown or mixed weather). Experiments on extensive restoration scenarios show our superior performance (see Fig. 1). The source code will be made available.