mlsanityData to 14 Oct 2026

Language-driven All-in-one Adverse Weather Removal

arXiv 2312.01381 ↗First posted 3 Dec 2023cs.CVcited by 36

On the map: Mixture-of-experts → Vision mixture-of-experts

Abstract (arXiv)

All-in-one (AiO) frameworks restore various adverse weather degradations with a single set of networks jointly. To handle various weather conditions, an AiO framework is expected to adaptively learn weather-specific knowledge for different degradations and shared knowledge for common patterns. However, existing methods: 1) rely on extra supervision signals, which are usually unknown in real-world applications; 2) employ fixed network structures, which restrict the diversity of weather-specific knowledge. In this paper, we propose a Language-driven Restoration framework (LDR) to alleviate the aforementioned issues. First, we leverage the power of pre-trained vision-language (PVL) models to enrich the diversity of weather-specific knowledge by reasoning about the occurrence, type, and severity of degradation, generating description-based degradation priors. Then, with the guidance of degradation prior, we sparsely select restoration experts from a candidate list dynamically based on a Mixture-of-Experts (MoE) structure. This enables us to adaptively learn the weather-specific and shared knowledge to handle various weather conditions (e.g., unknown or mixed weather). Experiments on extensive restoration scenarios show our superior performance (see Fig. 1). The source code will be made available.

Key facts. Cited by 36 papers in the map (data through 14 Oct 2026). Placed in Vision mixture-of-experts, one of 12 directions in Mixture-of-experts. Builds on 36 of its 59 references that are in the map. First posted on arXiv 3 Dec 2023.

Builds on

36 of 59 references.

References · 36

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2023Visual Instruction Tuningcited by 7,869
  3. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,137
  4. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,322
  5. 2017Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layercited by 3,509
  6. 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,711
  7. 2021Restormer: Efficient Transformer for High-Resolution Image Restorationcited by 1,627
  8. 2022Planning-oriented Autonomous Drivingcited by 1,087
  9. 2021GLaM: Efficient Scaling of Language Models with Mixture-of-Expertscited by 897
  10. 2020Pre-Trained Image Processing Transformercited by 893
  11. 2021Uformer: A General U-Shaped Transformer for Image Restorationcited by 866
  12. 2021Multi-Stage Progressive Image Restorationcited by 816
24 more
  1. 2023LISA: Reasoning Segmentation via Large Language Modelcited by 776
  2. 2022Simple Baselines for Image Restorationcited by 708
  3. 2019FFA-Net: Feature Fusion Attention Network for Single Image Dehazingcited by 307
  4. 2022Vision Transformers for Single Image Dehazingcited by 260
  5. 2021TransWeather: Transformer-based Restoration of Images Degraded by Adverse Weather Conditionscited by 244
  6. 2021HINet: Half Instance Normalization Network for Image Restorationcited by 237
  7. 2022Restoring Vision in Adverse Weather Conditions with Patch-Based Denoising Diffusion Modelscited by 215
  8. 2023Learning A Sparse Transformer Network for Effective Image Derainingcited by 205
  9. 2017Attentive Generative Adversarial Network for Raindrop Removal from a Single Imagecited by 204
  10. 2017DesnowNet: Context-Aware Deep Network for Snow Removalcited by 200
  11. 2019Heavy Rain Image Restoration: Integrating Physics Model and Conditional Adversarial Learningcited by 157
  12. 2023Efficient and Explicit Modelling of Image Hierarchies for Image Restorationcited by 145
  13. 2018Bringing a Blurry Frame Alive at High Frame-Rate with an Event Cameracited by 141
  14. 2023Iterative Prompt Learning for Unsupervised Backlit Image Enhancementcited by 107
  15. 2023Neural Map Prior for Autonomous Drivingcited by 68
  16. 2023Event Camera Data Pre-trainingcited by 50
  17. 2022MoEBERT: from BERT to Mixture-of-Experts via Importance-Guided Adaptationcited by 37
  18. 2022DRT: A Lightweight Single Image Deraining Recursive Transformercited by 26
  19. 2019Joint Stereo Video Deblurring, Scene Flow Estimation and Moving Object Segmentationcited by 11
  20. 2022BotBuster: Multi-platform Bot Detection Using A Mixture of Expertscited by 11
  21. 2023LDP: Language-driven Dual-Pixel Image Defocus Deblurring Networkcited by 11
  22. 2022INFWIDE: Image and Feature Space Wiener Deconvolution Network for Non-blind Image Deblurring in Low-Light Conditionscited by 8
  23. 2023Nordic Vehicle Dataset (NVD): Performance of vehicle detectors using newly captured NVD from UAV in different snowy weather conditionscited by 6
  24. 2023Surveillance Face Presentation Attack Detection Challengecited by 5

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2024: 11; 2025: 16; 2026: 9.

36 papers.

Sort:

By area:

  1. 2024AdaIR: Adaptive All-in-One Image Restoration via Frequency Mining and Modulationcited by 111
  2. 2024Complexity Experts are Task-Discriminative Learners for Any Image Restorationcited by 77
  3. 2024A Survey on All-in-One Image Restoration: Taxonomy, Evaluation and Future Trendscited by 60
  4. 2024Multi-Expert Adaptive Selection: Task-Balancing for All-in-One Image Restorationcited by 18
  5. 2024LoRA-IR: Taming Low-Rank Experts for Efficient All-in-One Image Restorationcited by 18
  6. 2024Diff-Restorer: Unleashing Visual Prompts for Diffusion-based Universal Image Restorationcited by 15
  7. 2025DGSolver: Diffusion Generalist Solver with Universal Posterior Sampling for Image Restorationcited by 5
  8. 2025ClusIR: Towards Cluster-Guided All-in-One Image Restorationcited by 5
  9. 2025Prompt to Restore, Restore to Prompt: Cyclic Prompting for Universal Adverse Weather Removalcited by 4
  10. 2024Adaptive Blind All-in-One Image Restorationcited by 3
  11. 2024LINK: Adaptive Modality Interaction for Audio-Visual Video Parsingcited by 3
  12. 2025Elucidating and Endowing the Diffusion Training Paradigm for General Image Restorationcited by 3
24 more
  1. 2025MUG: Pseudo Labeling Augmented Audio-Visual Mamba Network for Audio-Visual Video Parsingcited by 3
  2. 2025Grounding Degradations in Natural Language for All-In-One Video Restorationcited by 3
  3. 2025Robust Adverse Weather Removal via Spectral-based Spatial Groupingcited by 3
  4. 2025AWM-Fuse: Multi-Modality Image Fusion for Adverse Weather via Global and Local Text Perceptioncited by 3
  5. 2024LMHaze: Intensity-aware Image Dehazing with a Large-scale Multi-intensity Real Haze Datasetcited by 2
  6. 2025Clear Nights Ahead: Towards Multi-Weather Nighttime Image Restorationcited by 2
  7. 2025Real-World Adverse Weather Image Restoration via Dual-Level Reinforcement Learning with High-Quality Cold Startcited by 2
  8. 2026All-in-One Video Restoration under Smoothly Evolving Unknown Weather Degradationscited by 2
  9. 2024Accurate Forgetting for All-in-One Image Restoration Modelcited by 1
  10. 2025A Preliminary Study on GPT-Image Generation Model for Image Restorationcited by 1
  11. 2025TAP: Parameter-efficient Task-Aware Prompting for Adverse Weather Removalcited by 1
  12. 2025Clear Roads, Clear Vision: Advancements in Multi-Weather Restoration for Smart Transportationcited by 1
  13. 2026Degradation-Aware Prompt Learning with Cross-Modal Compensation for Adverse Weather Removalcited by 1
  14. 2024Learning Dual Transformers for All-In-One Image Restoration from a Frequency Perspectivecited by 0
  15. 2025MdaIF: Robust One-Stop Multi-Degradation-Aware Image Fusion with Language-Driven Semanticscited by 0
  16. 2025VLM-Augmented Degradation Modeling for Image Restoration Under Adverse Weather Conditionscited by 0
  17. 2025Unleashing Degradation-Carrying Features in Symmetric U-Net: Simpler and Stronger Baselines for All-in-One Image Restorationcited by 0
  18. 2026DELNet: Continuous All-in-One Weather Removal via Dynamic Expert Librarycited by 0
  19. 2026Seeing Roads Through Words: A Language-Guided Framework for RGB-T Driving Scene Segmentationcited by 0
  20. 2026LiDAR-Anchored Collaborative Distillation for Robust 2D Representationscited by 0
  21. 2026Leveraging Multimodal Large Language Models for All-in-One Image Restoration via a Mixture of Frequency Expertscited by 0
  22. 2026DDTNet: Degradation Disentanglement and Transfer Network for Test-Time All-in-One De-weathering Adaptationcited by 0
  23. 2026What to Remove, What to Preserve: Dual-Ambiguity Rectification for All-in-One Image Restorationcited by 0
  24. 2026Beyond Uniform Restoration: Empowering All-in-One Restoration with Pixel-Level Multimodal Guidancecited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2024Towards Real-World Adverse Weather Image Restoration: Enhancing Clearness and Semantics with Vision-Language Models
  2. 2024Continual All-in-One Adverse Weather Removal with Knowledge Replay on a Unified Network Structure
  3. 2025DA2Diff: Exploring Degradation-aware Adaptive Diffusion Priors for All-in-One Weather Restoration
  4. 2022Towards Efficient Single Image Dehazing and Desnowing
  5. 2026PVRF: All-in-one Adverse Weather Removal via Prior-modulated and Velocity-constrained Rectified Flow
  6. 2024SemiDDM-Weather: A Semi-supervised Learning Framework for All-in-one Adverse Weather Removal
  7. 2023WM-MoE: Weather-aware Multi-scale Mixture-of-Experts for Blind Adverse Weather Removal
  8. 2024All-in-one Weather-degraded Image Restoration via Adaptive Degradation-aware Self-prompting Model