# Feature Distillation: DNN-Oriented JPEG Compression Against Adversarial Examples

arXiv: https://arxiv.org/abs/1803.05787 · HTML: https://mlsanity.com/paper/1803.05787/

## Key facts

- Cited by 99 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Mar 2018.

## Built on by (most cited first, 30 of 99)

- [Enhancing the Transferability of Adversarial Attacks through Variance Tuning](https://mlsanity.com/paper/2103.15571/) (2021)
- [Admix: Enhancing the Transferability of Adversarial Attacks](https://mlsanity.com/paper/2102.00436/) (2021)
- [Advances in adversarial attacks and defenses in computer vision: A survey](https://mlsanity.com/paper/2108.00401/) (2021)
- [Improving Adversarial Transferability via Neuron Attribution-Based Attacks](https://mlsanity.com/paper/2204.00008/) (2022)
- [The effectiveness of feature attribution methods and its correlation with automatic evaluation scores](https://mlsanity.com/paper/2105.14944/) (2021)
- [Stochastic Variance Reduced Ensemble Adversarial Attack for Boosting the Adversarial Transferability](https://mlsanity.com/paper/2111.10752/) (2021)
- [Improving Vision Transformers by Revisiting High-frequency Components](https://mlsanity.com/paper/2204.00993/) (2022)
- [On Generating Transferable Targeted Perturbations](https://mlsanity.com/paper/2103.14641/) (2021)
- [Boosting Adversarial Transferability by Block Shuffle and Rotation](https://mlsanity.com/paper/2308.10299/) (2023)
- [Structure Invariant Transformation for better Adversarial Transferability](https://mlsanity.com/paper/2309.14700/) (2023)
- [Boosting Adversarial Transferability through Enhanced Momentum](https://mlsanity.com/paper/2103.10609/) (2021)
- [NICGSlowDown: Evaluating the Efficiency Robustness of Neural Image Caption Generation Models](https://mlsanity.com/paper/2203.15859/) (2022)
- [Boosting Adversarial Transferability by Achieving Flat Local Maxima](https://mlsanity.com/paper/2306.05225/) (2023)
- [MetaCloak: Preventing Unauthorized Subject-driven Text-to-image Diffusion-based Synthesis via Meta-learning](https://mlsanity.com/paper/2311.13127/) (2023)
- [An Adaptive Model Ensemble Adversarial Attack for Boosting Adversarial Transferability](https://mlsanity.com/paper/2308.02897/) (2023)
- [Biometrics: Trust, but Verify](https://mlsanity.com/paper/2105.06625/) (2021)
- [Improving the Transferability of Adversarial Samples by Path-Augmented Method](https://mlsanity.com/paper/2303.15735/) (2023)
- [Inspect, Understand, Overcome: A Survey of Practical Methods for AI Safety](https://mlsanity.com/paper/2104.14235/) (2021)
- [Transferable Adversarial Attack based on Integrated Gradients](https://mlsanity.com/paper/2205.13152/) (2022)
- [DISCO: Adversarial Defense with Local Implicit Functions](https://mlsanity.com/paper/2212.05630/) (2022)
- [Diversifying the High-level Features for better Adversarial Transferability](https://mlsanity.com/paper/2304.10136/) (2023)
- [SpectralDefense: Detecting Adversarial Attacks on CNNs in the Fourier Domain](https://mlsanity.com/paper/2103.03000/) (2021)
- [Boosting the Transferability of Adversarial Attacks with Global Momentum Initialization](https://mlsanity.com/paper/2211.11236/) (2022)
- [Adversarial Illusions in Multi-Modal Embeddings](https://mlsanity.com/paper/2308.11804/) (2023)
- [RFLA: A Stealthy Reflected Light Adversarial Attack in the Physical World](https://mlsanity.com/paper/2307.07653/) (2023)
- [Learning to Transform Dynamically for Better Adversarial Transferability](https://mlsanity.com/paper/2405.14077/) (2024)
- [Adversarial Attacks of Vision Tasks in the Past 10 Years: A Survey](https://mlsanity.com/paper/2410.23687/) (2024)
- [Adaptive Image Transformations for Transfer-based Adversarial Attack](https://mlsanity.com/paper/2111.13844/) (2021)
- [On the Robustness of Latent Diffusion Models](https://mlsanity.com/paper/2306.08257/) (2023)
- [Watermark Vaccine: Adversarial Attacks to Prevent Watermark Removal](https://mlsanity.com/paper/2207.08178/) (2022)

## Abstract (arXiv)

Image compression-based approaches for defending against the adversarial-example attacks, which threaten the safety use of deep neural networks (DNN), have been investigated recently. However, prior works mainly rely on directly tuning parameters like compression rate, to blindly reduce image features, thereby lacking guarantee on both defense efficiency (i.e. accuracy of polluted images) and classification accuracy of benign images, after applying defense methods. To overcome these limitations, we propose a JPEG-based defensive compression framework, namely "feature distillation", to effectively rectify adversarial examples without impacting classification accuracy on benign data. Our framework significantly escalates the defense efficiency with marginal accuracy reduction using a two-step method: First, we maximize malicious features filtering of adversarial input perturbations by developing defensive quantization in frequency domain of JPEG compression or decompression, guided by a semi-analytical method; Second, we suppress the distortions of benign features to restore classification accuracy through a DNN-oriented quantization refine process. Our experimental results show that proposed "feature distillation" can significantly surpass the latest input-transformation based mitigations such as Quilting and TV Minimization in three aspects, including defense efficiency (improve classification accuracy from ~20% to ~90% on adversarial examples), accuracy of benign images after defense (≤1% accuracy degradation), and processing time per image (~259× Speedup). Moreover, our solution can also provide the best defense efficiency (~60% accuracy) against the recent adaptive attack with least accuracy reduction (~1%) on benign images when compared with other input-transformation based defense methods.