mlsanityData to 14 Oct 2026

Watch your Up-Convolution: CNN Based Generative Deep Neural Networks are Failing to Reproduce Spectral Distributions

arXiv 2003.01826 ↗First posted 3 Mar 2020cs.CV eess.IVcited by 280

Not on the map. Why?

Cited by 33 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Generative convolutional deep neural networks, e.g. popular GAN architectures, are relying on convolution based up-sampling methods to produce non-scalar outputs like images or video sequences. In this paper, we show that common up-sampling methods, i.e. known as up-convolution or transposed convolution, are causing the inability of such models to reproduce spectral distributions of natural training data correctly. This effect is independent of the underlying architecture and we show that it can be used to easily detect generated data like deepfakes with up to 100% accuracy on public benchmarks. To overcome this drawback of current generative models, we propose to add a novel spectral regularization term to the training optimization objective. We show that this approach not only allows to train spectral consistent GANs that are avoiding high frequency errors. Also, we show that a correct approximation of the frequency spectrum has positive effects on the training stability and output quality of generative networks.

Key facts. Cited by 280 papers in the map (data through 14 Oct 2026). First posted on arXiv 3 Mar 2020.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 39; 2022: 28; 2023: 51; 2024: 45; 2025: 61; 2026: 56.

280 papers.

Sort:

By area:

  1. 2023GenImage: A Million-Scale Benchmark for Detecting AI-Generated Imagecited by 321
  2. 2021Spatial-Phase Shallow Learning: Rethinking Face Forgery Detection in Frequency Domaincited by 244
  3. 2023Forgery-aware Adaptive Transformer for Generalizable Synthetic Image Detectioncited by 182
  4. 2021ResViT: Residual vision transformers for multi-modal medical image synthesiscited by 177
  5. 2023DeepfakeBench: A Comprehensive Benchmark of Deepfake Detectioncited by 166
  6. 2023Raising the Bar of AI-generated Image Detection with CLIPcited by 164
  7. 2021Exploring Temporal Coherence for More General Video Face Forgery Detectioncited by 159
  8. 2021Are GAN generated images easy to detect? A critical analysis of the state-of-the-artcited by 139
  9. 2022Leveraging Real Talking Faces via Self-Supervision for Robust Forgery Detectioncited by 130
  10. 2021Pros and Cons of GAN Evaluation Measures: New Developmentscited by 123
  11. 2023Intriguing properties of synthetic images: from generative adversarial networks to diffusion modelscited by 123
  12. 2023PatchCraft: Exploring Texture Patch for Efficient AI-generated Image Detectioncited by 122
38 more
  1. 2022Towards the Detection of Diffusion Model Deepfakescited by 111
  2. 2021Face Forensics in the Wildcited by 87
  3. 2023Self-Supervised Video Forensics by Audio-Visual Anomaly Detectioncited by 85
  4. 2021Deepfakes Generation and Detection: State-of-the-art, open challenges, countermeasures, and way forwardcited by 82
  5. 2021WaveFake: A Data Set to Facilitate Audio Deepfake Detectioncited by 78
  6. 2022FrePGAN: Robust Deepfake Detection Using Frequency-level Perturbationscited by 75
  7. 2021A Closer Look at Fourier Spectrum Discrepancies for CNN-generated Images Detectioncited by 68
  8. 2024A Bias-Free Training Paradigm for More General AI-generated Image Detectioncited by 67
  9. 2021BiHPF: Bilateral High-Pass Filters for Robust Deepfake Detectioncited by 60
  10. 2021On the Frequency Bias of Generative Modelscited by 60
  11. 2024Any-Resolution AI-Generated Image Detection by Spectral Learningcited by 57
  12. 2021Countering Malicious DeepFakes: Survey, Battleground, and Horizoncited by 56
  13. 2023WOUAF: Weight Modulation for User Attribution and Fingerprinting in Text-to-Image Diffusion Modelscited by 53
  14. 2024Real-Time Deepfake Detection in the Real-Worldcited by 51
  15. 2023Beyond the Prior Forgery Knowledge: Mining Critical Clues for General Face Forgery Detectioncited by 50
  16. 2024Zero-Shot Detection of AI-Generated Imagescited by 50
  17. 2024Community Forensics: Using Thousands of Generators to Train Fake Image Detectorscited by 50
  18. 2023Generalizable Synthetic Image Detection via Language-guided Contrastive Learningcited by 49
  19. 2024FreqBlender: Enhancing DeepFake Detection by Blending Frequency Knowledgecited by 48
  20. 2022Deepfake Network Architecture Attributioncited by 47
  21. 2021Beyond the Spectrum: Detecting Deepfakes via Re-Synthesiscited by 44
  22. 2023SepMark: Deep Separable Watermarking for Unified Source Tracing and Deepfake Detectioncited by 44
  23. 2024A Single Simple Patch is All You Need for AI-generated Image Detectioncited by 44
  24. 2024Fake or JPEG? Revealing Common Biases in Generated Image Detection Datasetscited by 43
  25. 2024Is Synthetic Data all We Need? Benchmarking the Robustness of Models Trained with Synthetic Imagescited by 41
  26. 2021Spectral Unsupervised Domain Adaptation for Visual Recognitioncited by 40
  27. 2023Evading DeepFake Detectors via Adversarial Statistical Consistencycited by 37
  28. 2022Learning Second Order Local Anomaly for General Face Forgery Detectioncited by 36
  29. 2023Progressive Open Space Expansion for Open-Set Model Attributioncited by 32
  30. 2024On the Challenges and Opportunities in Generative AIcited by 31
  31. 2021SpectralDefense: Detecting Adversarial Attacks on CNNs in the Fourier Domaincited by 30
  32. 2021Dual Contrastive Loss and Attention for GANscited by 30
  33. 2023Deep Image Fingerprint: Towards Low Budget Synthetic Image Detection and Model Lineage Analysiscited by 30
  34. 2023Parents and Children: Distinguishing Multimodal DeepFakes from Natural Imagescited by 29
  35. 2021Robust Attentive Deep Neural Network for Exposing GAN-generated Facescited by 28
  36. 2022FrequencyLowCut Pooling -- Plug & Play against Catastrophic Overfittingcited by 28
  37. 2023Frequency Domain-based Dataset Distillationcited by 28
  38. 2024How to Trace Latent Generative Model Generated Images without Artificial Watermark?cited by 26

50 most cited of 280 All 280 (JSON)