mlsanityData to 14 Oct 2026

On the Effectiveness of Spectral Discriminators for Perceptual Quality Improvement

arXiv 2307.12027 ↗First posted 22 Jul 2023cs.CV eess.IVcited by 8

Not on the map. Why?

Abstract (arXiv)

Several recent studies advocate the use of spectral discriminators, which evaluate the Fourier spectra of images for generative modeling. However, the effectiveness of the spectral discriminators is not well interpreted yet. We tackle this issue by examining the spectral discriminators in the context of perceptual image super-resolution (i.e., GAN-based SR), as SR image quality is susceptible to spectral changes. Our analyses reveal that the spectral discriminator indeed performs better than the ordinary (a.k.a. spatial) discriminator in identifying the differences in the high-frequency range; however, the spatial discriminator holds an advantage in the low-frequency range. Thus, we suggest that the spectral and spatial discriminators shall be used simultaneously. Moreover, we improve the spectral discriminators by first calculating the patch-wise Fourier spectrum and then aggregating the spectra by Transformer. We verify the effectiveness of the proposed method twofold. On the one hand, thanks to the additional spectral discriminator, our obtained SR images have their spectra better aligned to those of the real images, which leads to a better PD tradeoff. On the other hand, our ensembled discriminator predicts the perceptual quality more accurately, as evidenced in the no-reference image quality assessment task.

Key facts. Cited by 8 papers in the map (data through 14 Oct 2026). Builds on 58 of its 80 references that are in the map. First posted on arXiv 22 Jul 2023.

Builds on

58 of 80 references.

References · 58

  1. 2017Attention Is All You Needcited by 60,361
  2. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
  3. 2015U-Net: Convolutional Networks for Biomedical Image Segmentationcited by 19,713
  4. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,364
  5. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,177
  6. 2016Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Networkcited by 1,908
  7. 2021SwinIR: Image Restoration Using Swin Transformercited by 1,887
  8. 2021Restormer: Efficient Transformer for High-Resolution Image Restorationcited by 1,627
  9. 2017Enhanced Deep Residual Networks for Single Image Super-Resolutioncited by 1,552
  10. 2021MLP-Mixer: An all-MLP Architecture for Visioncited by 1,446
  11. 2018On the Spectral Bias of Neural Networkscited by 1,341
  12. 2018Image Super-Resolution Using Very Deep Residual Channel Attention Networkscited by 1,264
38 more
  1. 2018ESRGAN: Enhanced Super-Resolution Generative Adversarial Networkscited by 1,254
  2. 2014Image Super-Resolution Using Deep Convolutional Networkscited by 1,224
  3. 2021Real-ESRGAN: Training Real-World Blind Super-Resolution with Pure Synthetic Datacited by 970
  4. 2015Accurate Image Super-Resolution Using Very Deep Convolutional Networkscited by 903
  5. 2019CNN-generated images are surprisingly easy to spot... for nowcited by 847
  6. 2021Multi-Stage Progressive Image Restorationcited by 816
  7. 2018Residual Dense Network for Image Super-Resolutioncited by 799
  8. 2017The Perception-Distortion Tradeoffcited by 587
  9. 2020Leveraging Frequency Analysis for Deep Fake Image Recognitioncited by 504
  10. 2021Intriguing Properties of Vision Transformerscited by 446
  11. 2018Recovering Realistic Texture in Image Super-resolution by Deep Spatial Feature Transformcited by 371
  12. 2019Deep Learning for Image Super-resolution: A Surveycited by 369
  13. 2019KonIQ-10k: An ecologically valid database for deep learning of blind image quality assessmentcited by 355
  14. 2020Learning Enriched Features for Real Image Restoration and Enhancementcited by 312
  15. 2019High Frequency Component Helps Explain the Generalization of Convolutional Neural Networkscited by 310
  16. 2022How Do Vision Transformers Work?cited by 307
  17. 2020Watch your Up-Convolution: CNN Based Generative Deep Neural Networks are Failing to Reproduce Spectral Distributionscited by 280
  18. 2015Massive Online Crowdsourced Study of Subjective and Objective Picture Qualitycited by 209
  19. 2020Focal Frequency Loss for Image Reconstruction and Synthesiscited by 206
  20. 2016EnhanceNet: Single Image Super-Resolution Through Automated Texture Synthesiscited by 178
  21. 2020PIPAL: a Large-Scale Image Quality Assessment Dataset for Perceptual Image Restorationcited by 178
  22. 2017RankIQA: Learning from Rankings for No-reference Image Quality Assessmentcited by 151
  23. 2019RankSRGAN: Generative Adversarial Networks with Ranker for Image Super-Resolutioncited by 145
  24. 2019Fourier Spectrum Discrepancies in Deep Network Generated Imagescited by 139
  25. 2020Interpreting Super-Resolution Networks with Local Attribution Mapscited by 136
  26. 2020Structure-Preserving Super Resolution with Gradient Guidancecited by 116
  27. 2022Details or Artifacts: A Locally Discriminative Learning Approach to Realistic Image Super-Resolutioncited by 114
  28. 2021On the Adversarial Robustness of Vision Transformerscited by 106
  29. 2021Image Quality Assessment using Contrastive Learningcited by 95
  30. 2019dipIQ: Blind Image Quality Assessment by Learning-to-Rank Discriminable Image Pairscited by 73
  31. 2021Fourier Space Losses for Efficient Perceptual Image Super-Resolutioncited by 73
  32. 2021A Closer Look at Fourier Spectrum Discrepancies for CNN-generated Images Detectioncited by 68
  33. 2021Burst Image Restoration and Enhancementcited by 64
  34. 2021On the Frequency Bias of Generative Modelscited by 60
  35. 2021Adversarial Robustness Comparison of Vision Transformer and MLP-Mixer to CNNscited by 57
  36. 2023Masked Image Training for Generalizable Deep Image Denoisingcited by 57
  37. 2019On the Effectiveness of Low Frequency Perturbationscited by 56
  38. 2019Exploiting Unlabeled Data in CNNs by Self-supervised Learning to Rankcited by 52

50 most cited of 58 All 58 (JSON)

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2023: 2; 2024: 3; 2025: 2; 2026: 1.

8 papers.

  1. 2024SeD: Semantic-Aware Discriminator for Image Super-Resolutioncited by 26
  2. 2024Training Generative Image Super-Resolution Models by Wavelet-Domain Losses Enables Better Control of Artifactscited by 16
  3. 2023Perception-Distortion Balanced Super-Resolution: A Multi-Objective Optimization Perspectivecited by 11
  4. 2024Training Transformer Models by Wavelet Losses Improves Quantitative and Visual Performance in Single Image Super-Resolutioncited by 10
  5. 2023Generalized Lightness Adaptation with Channel Selective Normalizationcited by 4
  6. 2025Exploring Semantic Feature Discrimination for Perceptual Image Super-Resolution and Opinion-Unaware No-Reference Image Quality Assessmentcited by 3
  7. 2025Latent Posterior-Mean Rectified Flow for Higher-Fidelity Perceptual Face Restorationcited by 0
  8. 2026HiTokSR: A Coarse-to-Fine Tokenizer with Hierarchical Codebooks for High-Fidelity Real-World Image Super-Resolutioncited by 0