mlsanityData to 7 Oct 2026

Learn from Unpaired Data for Image Restoration: A Variational Bayes Approach

arXiv 2204.10090 ↗First posted 21 Apr 2022eess.IV cs.CVcited by 18

Not on the map. Why?

Abstract (arXiv)

Collecting paired training data is difficult in practice, but the unpaired samples broadly exist. Current approaches aim at generating synthesized training data from unpaired samples by exploring the relationship between the corrupted and clean data. This work proposes LUD-VAE, a deep generative method to learn the joint probability density function from data sampled from marginal distributions. Our approach is based on a carefully designed probabilistic graphical model in which the clean and corrupted data domains are conditionally independent. Using variational inference, we maximize the evidence lower bound (ELBO) to estimate the joint probability density function. Furthermore, we show that the ELBO is computable without paired samples under the inference invariant assumption. This property provides the mathematical rationale of our approach in the unpaired setting. Finally, we apply our method to real-world image denoising, super-resolution, and low-light image enhancement tasks and train the models using the synthetic data generated by the LUD-VAE. Experimental results validate the advantages of our method over other approaches.

Key facts. Cited by 18 papers in the map (data through 7 Oct 2026). Builds on 63 of its 92 references that are in the map. First posted on arXiv 21 Apr 2022.

Builds on

63 of 92 references.

References · 63

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  3. 2022Generative Adversarial Networkscited by 5,603
  4. 2017Neural Discrete Representation Learningcited by 4,650
  5. 2013Auto-Encoding Variational Bayescited by 3,980
  6. 2016Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Networkcited by 1,898
  7. 2021SwinIR: Image Restoration Using Swin Transformercited by 1,869
  8. 2021Restormer: Efficient Transformer for High-Resolution Image Restorationcited by 1,618
  9. 2019Generating Diverse High-Fidelity Images with VQ-VAE-2cited by 1,376
  10. 2016Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoisingcited by 1,352
  11. 2018Image Super-Resolution Using Very Deep Residual Channel Attention Networkscited by 1,256
  12. 2018ESRGAN: Enhanced Super-Resolution Generative Adversarial Networkscited by 1,250
38 more
  1. 2017Deep Image Priorcited by 1,097
  2. 2021Uformer: A General U-Shaped Transformer for Image Restorationcited by 864
  3. 2016Conditional Image Generation with PixelCNN Decoderscited by 859
  4. 2018Deep Retinex Decomposition for Low-Light Enhancementcited by 720
  5. 2020NVAE: A Deep Hierarchical Variational Autoencodercited by 635
  6. 2015Generating Sentences from a Continuous Spacecited by 607
  7. 2020Zero-Reference Deep Curve Estimation for Low-Light Image Enhancementcited by 541
  8. 2018Noise2Noise: Learning Image Restoration without Clean Datacited by 526
  9. 2017FFDNet: Toward a Fast and Flexible Solution for CNN based Image Denoisingcited by 509
  10. 2020Plug-and-Play Image Restoration with Deep Denoiser Priorcited by 467
  11. 2019EnlightenGAN: Deep Light Enhancement without Paired Supervisioncited by 466
  12. 2017Towards Principled Methods for Training Generative Adversarial Networkscited by 460
  13. 2019Toward Real-World Single Image Super-Resolution: A New Benchmark and A New Modelcited by 400
  14. 2016Accelerating the Super-Resolution Convolutional Neural Networkcited by 393
  15. 2018Noise2Void - Learning Denoising from Single Noisy Imagescited by 359
  16. 2016Ladder Variational Autoencoderscited by 315
  17. 2020Retinex-inspired Unrolling with Cooperative Prior Architecture Search for Low-light Image Enhancementcited by 303
  18. 2018Toward Convolutional Blind Denoising of Real Photographscited by 274
  19. 2019Noise2Self: Blind Denoising by Self-Supervisioncited by 274
  20. 2020Very Deep VAEs Generalize Autoregressive Models and Can Outperform Them on Imagescited by 261
  21. 2016Generating Images with Perceptual Similarity Metrics based on Deep Networkscited by 221
  22. 2017Benchmarking Denoising Algorithms with Real Photographscited by 209
  23. 2020SRFlow: Learning the Super-Resolution Space with Normalizing Flowcited by 209
  24. 2021Low-Light Image Enhancement with Normalizing Flowcited by 184
  25. 2019Real Image Denoising with Feature Attentioncited by 176
  26. 2019Blind Super-Resolution Kernel Estimation using an Internal-GANcited by 175
  27. 2019High-Quality Self-Supervised Deep Image Denoisingcited by 161
  28. 2018Unsupervised Image Super-Resolution using Cycle-in-Cycle Generative Adversarial Networkscited by 133
  29. 2019Frequency Separation for Real-World Super-Resolutioncited by 115
  30. 2019Noise Flow: Noise Modeling with Conditional Normalizing Flowscited by 108
  31. 2020Dual Adversarial Network: Toward Real-world Noise Removal and Noise Generationcited by 101
  32. 2018To learn image super-resolution, use a GAN to learn how to do image degradation firstcited by 94
  33. 2019Robust and interpretable blind image denoising via bias-free convolutional neural networkscited by 87
  34. 2020Unsupervised Real-world Image Super Resolution via Domain-distance Aware Trainingcited by 83
  35. 2020NTIRE 2020 Challenge on Real-World Image Super-Resolution: Methods and Resultscited by 83
  36. 2020Unpaired Learning of Deep Image Denoisingcited by 74
  37. 2019Unsupervised Learning for Real-World Super-Resolutioncited by 64
  38. 2017CycleGAN, a Master of Steganographycited by 63

50 most cited of 63 All 63 (JSON)

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 2; 2023: 3; 2024: 8; 2025: 3; 2026: 2.

18 papers.

Sort:

By area:

  1. 2023Low-Light Image Enhancement with Wavelet-based Diffusion Modelscited by 126
  2. 2024LightenDiffusion: Unsupervised Low-Light Image Enhancement with Latent-Retinex Diffusion Modelscited by 70
  3. 2024Transfer CLIP for Generalizable Image Denoisingcited by 29
  4. 2023Score Priors Guided Deep Variational Inference for Unsupervised Real-World Single Image Denoisingcited by 15
  5. 2022Noise Self-Regression: A New Learning Paradigm to Enhance Low-Light Images Without Task-Related Datacited by 12
  6. 2024Masked and Shuffled Blind Spot Denoising for Real-World Imagescited by 12
  7. 2024Unpaired Photo-realistic Image Deraining with Energy-informed Diffusion Modelcited by 11
  8. 2025Reversing Flow for Image Restorationcited by 10
  9. 2024Positive2Negative: Breaking the Information-Lossy Barrier in Self-Supervised Single Image Denoisingcited by 6
  10. 2022Practical exposure correction via compensationcited by 5
  11. 2024Step-Calibrated Diffusion for Biomedical Optical Image Restorationcited by 5
  12. 2024SeNM-VAE: Semi-Supervised Noise Modeling with Hierarchical Variational Autoencodercited by 4
6 more
  1. 2024Beyond Image Prior: Embedding Noise Prior into Conditional Denoising Transformercited by 3
  2. 2023Addressing preferred orientation in single-particle cryo-EM through AI-generated auxiliary particlescited by 0
  3. 2025PEdger++: Practical Edge Detection via Assembling Cross Informationcited by 0
  4. 2025QWD-GAN: Quality-aware Wavelet-driven GAN for Unsupervised Medical Microscopy Images Denoisingcited by 0
  5. 2026Beyond Strict Pairing: Arbitrarily Paired Training for High-Performance Infrared and Visible Image Fusioncited by 0
  6. 2026Unpaired Joint Distribution Modeling via Multi-Scale Image Representationscited by 0