# ADD: Frequency Attention and Multi-View based Knowledge Distillation to Detect Low-Quality Compressed Deepfake Images

arXiv: https://arxiv.org/abs/2112.03553 · HTML: https://mlsanity.com/paper/2112.03553/

## Key facts

- Cited by 48 papers in the map (data through 14 Oct 2026).
- Placed in General knowledge distillation methods, one of 12 directions in Knowledge distillation.
- Builds on 34 of its 62 references that are in the map.
- First posted on arXiv 7 Dec 2021.

## Placed in

- Knowledge distillation / [General knowledge distillation methods](https://mlsanity.com/distill/general-knowledge-distillation-methods/)

## Builds on (most cited first, 30 of 34)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://mlsanity.com/paper/1905.11946/) (2019)
- [Non-local Neural Networks](https://mlsanity.com/paper/1711.07971/) (2017)
- [FitNets: Hints for Thin Deep Nets](https://mlsanity.com/paper/1412.6550/) (2014)
- [FaceForensics++: Learning to Detect Manipulated Facial Images](https://mlsanity.com/paper/1901.08971/) (2019)
- [Paying More Attention to Attention: Improving the Performance of Convolutional Neural Networks via Attention Transfer](https://mlsanity.com/paper/1612.03928/) (2016)
- [VGGFace2: A dataset for recognising faces across pose and age](https://mlsanity.com/paper/1710.08092/) (2017)
- [Contrastive Representation Distillation](https://mlsanity.com/paper/1910.10699/) (2019)
- [Encoding in Style: a StyleGAN Encoder for Image-to-Image Translation](https://mlsanity.com/paper/2008.00951/) (2020)
- [First Order Motion Model for Image Animation](https://mlsanity.com/paper/2003.00196/) (2020)
- [Face2Face: Real-time Face Capture and Reenactment of RGB Videos](https://mlsanity.com/paper/2007.14808/) (2020)
- [Thinking in Frequency: Face Forgery Detection by Mining Frequency-aware Clues](https://mlsanity.com/paper/2007.09355/) (2020)
- [Leveraging Frequency Analysis for Deep Fake Image Recognition](https://mlsanity.com/paper/2003.08685/) (2020)
- [Face X-ray for More General Face Forgery Detection](https://mlsanity.com/paper/1912.13458/) (2019)
- [Exposing DeepFake Videos By Detecting Face Warping Artifacts](https://mlsanity.com/paper/1811.00656/) (2018)
- [Relation Networks for Object Detection](https://mlsanity.com/paper/1711.11575/) (2017)
- [Detecting and Simulating Artifacts in GAN Fake Images](https://mlsanity.com/paper/1907.06515/) (2019)
- [Sliced Wasserstein Discrepancy for Unsupervised Domain Adaptation](https://mlsanity.com/paper/1903.04064/) (2019)
- [FaceShifter: Towards High Fidelity And Occlusion Aware Face Swapping](https://mlsanity.com/paper/1912.13457/) (2019)
- [SCA-CNN: Spatial and Channel-wise Attention in Convolutional Networks for Image Captioning](https://mlsanity.com/paper/1611.05594/) (2016)
- [Learning Deep Representations with Probabilistic Knowledge Transfer](https://mlsanity.com/paper/1803.10837/) (2018)
- [Training behavior of deep neural network in frequency domain](https://mlsanity.com/paper/1807.01251/) (2018)
- [Adversarial Latent Autoencoders](https://mlsanity.com/paper/2004.04467/) (2020)
- [Fourier Spectrum Discrepancies in Deep Network Generated Images](https://mlsanity.com/paper/1911.06465/) (2019)
- [Generative Modeling using the Sliced Wasserstein Distance](https://mlsanity.com/paper/1803.11188/) (2018)
- [Spatial Frequency Bias in Convolutional Generative Adversarial Networks](https://mlsanity.com/paper/2010.01473/) (2020)

## Built on by (most cited first, 30 of 48)

- [C2P-CLIP: Injecting Category Common Prompt in CLIP to Enhance Generalization in Deepfake Detection](https://mlsanity.com/paper/2408.09647/) (2024)
- [TALL: Thumbnail Layout for Deepfake Video Detection](https://mlsanity.com/paper/2307.07494/) (2023)
- [Deepfake Detection: A Comprehensive Survey from the Reliability Perspective](https://mlsanity.com/paper/2211.10881/) (2022)
- [FIRE: Robust Detection of Diffusion-Generated Images via Frequency-Guided Reconstruction Error](https://mlsanity.com/paper/2412.07140/) (2024)
- [Quality-Agnostic Deepfake Detection with Intra-model Collaborative Learning](https://mlsanity.com/paper/2309.05911/) (2023)
- [Learning Spatiotemporal Inconsistency via Thumbnail Layout for Face Deepfake Detection](https://mlsanity.com/paper/2403.10261/) (2024)
- [Frequency-Aware Deepfake Detection: Improving Generalizability through Frequency Space Learning](https://mlsanity.com/paper/2403.07240/) (2024)
- [GM-DF: Generalized Multi-Scenario Deepfake Detection](https://mlsanity.com/paper/2406.20078/) (2024)
- [Unified Frequency-Assisted Transformer Framework for Detecting and Grounding Multi-Modal Manipulation](https://mlsanity.com/paper/2309.09667/) (2023)
- [SoK: Systematization and Benchmarking of Deepfake Detectors in a Unified Framework](https://mlsanity.com/paper/2401.04364/) (2024)
- [A Survey of Defenses Against AI-Generated Visual Media: Detection,Disruption, and Authentication](https://mlsanity.com/paper/2407.10575/) (2024)
- [Why Do Facial Deepfake Detectors Fail?](https://mlsanity.com/paper/2302.13156/) (2023)
- [Data-Independent Operator: A Training-Free Artifact Representation Extractor for Generalizable Deepfake Detection](https://mlsanity.com/paper/2403.06803/) (2024)
- [Deepfake Media Generation and Detection in the Generative AI Era: A Survey and Outlook](https://mlsanity.com/paper/2411.19537/) (2024)
- [Diff-ID: An Explainable Identity Difference Quantification Framework for DeepFake Detection](https://mlsanity.com/paper/2303.18174/) (2023)
- [Evolving from Single-modal to Multi-modal Facial Deepfake Detection: Progress and Challenges](https://mlsanity.com/paper/2406.06965/) (2024)
- [ODDN: Addressing Unpaired Data Challenges in Open-World Deepfake Detection on Online Social Networks](https://mlsanity.com/paper/2410.18687/) (2024)
- [Spatial-Temporal Frequency Forgery Clue for Video Forgery Detection in VIS and NIR Scenario](https://mlsanity.com/paper/2207.01906/) (2022)
- [RetouchingFFHQ: A Large-scale Dataset for Fine-grained Face Retouching Detection](https://mlsanity.com/paper/2307.10642/) (2023)
- [Artifact Feature Purification for Cross-domain Detection of AI-generated Images](https://mlsanity.com/paper/2403.11172/) (2024)
- [UGAD: Universal Generative AI Detector utilizing Frequency Fingerprints](https://mlsanity.com/paper/2409.07913/) (2024)
- [Unleashing the Potential of Consistency Learning for Detecting and Grounding Multi-Modal Media Manipulation](https://mlsanity.com/paper/2506.05890/) (2025)
- [Compressed Deepfake Video Detection Based on 3D Spatiotemporal Trajectories](https://mlsanity.com/paper/2404.18149/) (2024)
- [Exploring the Impact of Moire Pattern on Deepfake Detectors](https://mlsanity.com/paper/2407.10399/) (2024)
- [Passive Deepfake Detection Across Multi-modalities: A Comprehensive Survey](https://mlsanity.com/paper/2411.17911/) (2024)
- [DuoCast: Duo-Probabilistic Diffusion for Precipitation Nowcasting](https://mlsanity.com/paper/2412.01091/) (2024)
- [LLMs Are Not Yet Ready for Deepfake Image Detection](https://mlsanity.com/paper/2506.10474/) (2025)
- [FakeChain: Exposing Shallow Cues in Multi-Step Deepfake Detection](https://mlsanity.com/paper/2509.16602/) (2025)
- [Vision Transformer Based Video Hashing Retrieval for Tracing the Source of Fake Videos](https://mlsanity.com/paper/2112.08117/) (2021)
- [Fighting Malicious Media Data: A Survey on Tampering Detection and Deepfake Detection](https://mlsanity.com/paper/2212.05667/) (2022)

## Abstract (arXiv)

Despite significant advancements of deep learning-based forgery detectors for distinguishing manipulated deepfake images, most detection approaches suffer from moderate to significant performance degradation with low-quality compressed deepfake images. Because of the limited information in low-quality images, detecting low-quality deepfake remains an important challenge. In this work, we apply frequency domain learning and optimal transport theory in knowledge distillation (KD) to specifically improve the detection of low-quality compressed deepfake images. We explore transfer learning capability in KD to enable a student network to learn discriminative features from low-quality images effectively. In particular, we propose the Attention-based Deepfake detection Distiller (ADD), which consists of two novel distillations: 1) frequency attention distillation that effectively retrieves the removed high-frequency components in the student network, and 2) multi-view attention distillation that creates multiple attention vectors by slicing the teacher's and student's tensors under different views to transfer the teacher tensor's distribution to the student more efficiently. Our extensive experimental results demonstrate that our approach outperforms state-of-the-art baselines in detecting low-quality compressed deepfake images.