# Detecting Adversarial Samples Using Influence Functions and Nearest Neighbors

arXiv: https://arxiv.org/abs/1909.06872 · HTML: https://mlsanity.com/paper/1909.06872/

## Key facts

- Cited by 72 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Sep 2019.

## Built on by (most cited first, 30 of 72)

- [Training Data Influence Analysis and Estimation: A Survey](https://mlsanity.com/paper/2212.04612/) (2022)
- [Advances in adversarial attacks and defenses in computer vision: A survey](https://mlsanity.com/paper/2108.00401/) (2021)
- [Adversarial Example Detection for DNN Models: A Review and Experimental Comparison](https://mlsanity.com/paper/2105.00203/) (2021)
- [LiBRe: A Practical Bayesian Approach to Adversarial Detection](https://mlsanity.com/paper/2103.14835/) (2021)
- [SpectralDefense: Detecting Adversarial Attacks on CNNs in the Fourier Domain](https://mlsanity.com/paper/2103.03000/) (2021)
- [Robust Models are less Over-Confident](https://mlsanity.com/paper/2210.05938/) (2022)
- [Understanding Influence Functions and Datamodels via Harmonic Analysis](https://mlsanity.com/paper/2210.01072/) (2022)
- [Membership Inference Attack Using Self Influence Functions](https://mlsanity.com/paper/2205.13680/) (2022)
- [Revisiting the Fragility of Influence Functions](https://mlsanity.com/paper/2303.12922/) (2023)
- [MagDR: Mask-guided Detection and Reconstruction for Defending Deepfakes](https://mlsanity.com/paper/2103.14211/) (2021)
- [Two Coupled Rejection Metrics Can Tell Adversarial Examples Apart](https://mlsanity.com/paper/2105.14785/) (2021)
- [Defenses in Adversarial Machine Learning: A Survey](https://mlsanity.com/paper/2312.08890/) (2023)
- [Multi-Expert Adversarial Attack Detection in Person Re-identification Using Context Inconsistency](https://mlsanity.com/paper/2108.09891/) (2021)
- [Out-of-Distribution Generalization Analysis via Influence Function](https://mlsanity.com/paper/2101.08521/) (2021)
- [Visual Adversarial Attacks and Defenses in the Physical World: A Survey](https://mlsanity.com/paper/2211.01671/) (2022)
- [Understanding Impact of Human Feedback via Influence Functions](https://mlsanity.com/paper/2501.05790/) (2025)
- [Detecting AutoAttack Perturbations in the Frequency Domain](https://mlsanity.com/paper/2111.08785/) (2021)
- [NoiLIn: Improving Adversarial Training and Correcting Stereotype of Noisy Labels](https://mlsanity.com/paper/2105.14676/) (2021)
- [Residue-Based Natural Language Adversarial Attack Detection](https://mlsanity.com/paper/2204.10192/) (2022)
- [Understanding Programmatic Weak Supervision via Source-aware Influence Function](https://mlsanity.com/paper/2205.12879/) (2022)
- [Dispersed Pixel Perturbation-based Imperceptible Backdoor Trigger for Image Classifier Models](https://mlsanity.com/paper/2208.09336/) (2022)
- [Special Session: Towards an Agile Design Methodology for Efficient, Reliable, and Secure ML Systems](https://mlsanity.com/paper/2204.09514/) (2022)
- [New Adversarial Image Detection Based on Sentiment Analysis](https://mlsanity.com/paper/2305.03173/) (2023)
- [Simple Post-Training Robustness Using Test Time Augmentations and Random Forest](https://mlsanity.com/paper/2109.08191/) (2021)
- [A Uniform Framework for Anomaly Detection in Deep Neural Networks](https://mlsanity.com/paper/2110.03092/) (2021)
- [Revisiting the Trade-off between Accuracy and Robustness via Weight Distribution of Filters](https://mlsanity.com/paper/2306.03430/) (2023)
- [Data Poisoning Attacks on Off-Policy Policy Evaluation Methods](https://mlsanity.com/paper/2404.04714/) (2024)
- [Anomaly Detection of Adversarial Examples using Class-conditional Generative Adversarial Networks](https://mlsanity.com/paper/2105.10101/) (2021)
- [Unity is strength: Improving the Detection of Adversarial Examples with Ensemble Approaches](https://mlsanity.com/paper/2111.12631/) (2021)
- [Nowhere to Hide: A Lightweight Unsupervised Detector against Adversarial Examples](https://mlsanity.com/paper/2210.08579/) (2022)

## Abstract (arXiv)

Deep neural networks (DNNs) are notorious for their vulnerability to adversarial attacks, which are small perturbations added to their input images to mislead their prediction. Detection of adversarial examples is, therefore, a fundamental requirement for robust classification frameworks. In this work, we present a method for detecting such adversarial attacks, which is suitable for any pre-trained neural network classifier. We use influence functions to measure the impact of every training sample on the validation set data. From the influence scores, we find the most supportive training samples for any given validation example. A k-nearest neighbor (k-NN) model fitted on the DNN's activation layers is employed to search for the ranking of these supporting training samples. We observe that these samples are highly correlated with the nearest neighbors of the normal inputs, while this correlation is much weaker for adversarial inputs. We train an adversarial detector using the k-NN ranks and distances and show that it successfully distinguishes adversarial examples, getting state-of-the-art results on six attack methods with three datasets. Code is available at https://github.com/giladcohen/NNIF_adv_defense.