# Unsupervised Restoration of Weather-affected Images using Deep Gaussian Process-based CycleGAN

arXiv: https://arxiv.org/abs/2204.10970 · HTML: https://mlsanity.com/paper/2204.10970/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 51 of its 86 references that are in the map.
- First posted on arXiv 23 Apr 2022.

## Builds on (most cited first, 30 of 51)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [Multi-Stage Progressive Image Restoration](https://mlsanity.com/paper/2102.02808/) (2021)
- [Unsupervised Image-to-Image Translation Networks](https://mlsanity.com/paper/1703.00848/) (2017)
- [Res2Net: A New Multi-scale Backbone Architecture](https://mlsanity.com/paper/1904.01169/) (2019)
- [Benchmarking Single Image Dehazing and Beyond](https://mlsanity.com/paper/1712.04143/) (2017)
- [Domain Adaptive Faster R-CNN for Object Detection in the Wild](https://mlsanity.com/paper/1803.03243/) (2018)
- [Semantic Foggy Scene Understanding with Synthetic Data](https://mlsanity.com/paper/1708.07819/) (2017)
- [Deep Joint Rain Detection and Removal from a Single Image](https://mlsanity.com/paper/1609.07769/) (2016)
- [DehazeNet: An End-to-End System for Single Image Haze Removal](https://mlsanity.com/paper/1601.07661/) (2016)
- [DualGAN: Unsupervised Dual Learning for Image-to-Image Translation](https://mlsanity.com/paper/1704.02510/) (2017)
- [Toward Multimodal Image-to-Image Translation](https://mlsanity.com/paper/1711.11586/) (2017)
- [Learning to Discover Cross-Domain Relations with Generative Adversarial Networks](https://mlsanity.com/paper/1703.05192/) (2017)
- [Progressive Image Deraining Networks: A Better and Simpler Baseline](https://mlsanity.com/paper/1901.09221/) (2019)
- [Strong-Weak Distribution Alignment for Adaptive Object Detection](https://mlsanity.com/paper/1812.04798/) (2018)
- [Multi-Scale Progressive Fusion Network for Single Image Deraining](https://mlsanity.com/paper/2003.10985/) (2020)
- [Density-aware Single Image De-raining using a Multi-stream Dense Network](https://mlsanity.com/paper/1802.07412/) (2018)
- [TransWeather: Transformer-based Restoration of Images Degraded by Adverse Weather Conditions](https://mlsanity.com/paper/2111.14813/) (2021)
- [GridDehazeNet: Attention-Based Multi-Scale Network for Image Dehazing](https://mlsanity.com/paper/1908.03245/) (2019)
- [Coupled Generative Adversarial Networks](https://mlsanity.com/paper/1606.07536/) (2016)
- [Recurrent Squeeze-and-Excitation Context Aggregation Net for Single Image Deraining](https://mlsanity.com/paper/1807.05698/) (2018)
- [Attentive Generative Adversarial Network for Raindrop Removal from a Single Image](https://mlsanity.com/paper/1711.10098/) (2017)
- [DesnowNet: Context-Aware Deep Network for Snow Removal](https://mlsanity.com/paper/1708.04512/) (2017)
- [Spatial Attentive Single-Image Deraining with a High Quality Real Rain Dataset](https://mlsanity.com/paper/1904.01538/) (2019)
- [Densely Connected Pyramid Dehazing Network](https://mlsanity.com/paper/1803.08396/) (2018)
- [Clearing the Skies: A deep network architecture for single-image rain removal](https://mlsanity.com/paper/1609.02087/) (2016)
- [Gated Fusion Network for Single Image Dehazing](https://mlsanity.com/paper/1804.00213/) (2018)
- [Uncertainty Guided Multi-Scale Residual Learning-using a Cycle Spinning CNN for Single Image De-Raining](https://mlsanity.com/paper/1906.11129/) (2019)
- [Single Image Deraining: A Comprehensive Benchmark Analysis](https://mlsanity.com/paper/1903.08558/) (2019)

## Built on by (most cited first, 2 of 2)

- [RoCA: Robust Cross-Domain End-to-End Autonomous Driving](https://mlsanity.com/paper/2506.10145/) (2025)
- [Re-boosting Self-Collaboration Parallel Prompt GAN for Unsupervised Image Restoration](https://mlsanity.com/paper/2408.09241/) (2024)

## Abstract (arXiv)

Existing approaches for restoring weather-degraded images follow a fully-supervised paradigm and they require paired data for training. However, collecting paired data for weather degradations is extremely challenging, and existing methods end up training on synthetic data. To overcome this issue, we describe an approach for supervising deep networks that are based on CycleGAN, thereby enabling the use of unlabeled real-world data for training. Specifically, we introduce new losses for training CycleGAN that lead to more effective training, resulting in high-quality reconstructions. These new losses are obtained by jointly modeling the latent space embeddings of predicted clean images and original clean images through Deep Gaussian Processes. This enables the CycleGAN architecture to transfer the knowledge from one domain (weather-degraded) to another (clean) more effectively. We demonstrate that the proposed method can be effectively applied to different restoration tasks like de-raining, de-hazing and de-snowing and it outperforms other unsupervised techniques (that leverage weather-based characteristics) by a considerable margin.