# Adversarial Logit Pairing

arXiv: https://arxiv.org/abs/1803.06373 · HTML: https://mlsanity.com/paper/1803.06373/

## Key facts

- Cited by 224 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 16 Mar 2018.

## Built on by (most cited first, 30 of 224)

- [Recent Advances in Adversarial Training for Adversarial Robustness](https://mlsanity.com/paper/2102.01356/) (2021)
- [Cross-Entropy Loss Functions: Theoretical Analysis and Applications](https://mlsanity.com/paper/2304.07288/) (2023)
- [Fixing Data Augmentation to Improve Adversarial Robustness](https://mlsanity.com/paper/2103.01946/) (2021)
- [Data Augmentation Can Improve Robustness](https://mlsanity.com/paper/2111.05328/) (2021)
- [The Values Encoded in Machine Learning Research](https://mlsanity.com/paper/2106.15590/) (2021)
- [Defensive Unlearning with Adversarial Training for Robust Concept Erasure in Diffusion Models](https://mlsanity.com/paper/2405.15234/) (2024)
- [Robustness and Accuracy Could Be Reconcilable by (Proper) Definition](https://mlsanity.com/paper/2202.10103/) (2022)
- [LAS-AT: Adversarial Training with Learnable Attack Strategy](https://mlsanity.com/paper/2203.06616/) (2022)
- [Style-Hallucinated Dual Consistency Learning for Domain Generalized Semantic Segmentation](https://mlsanity.com/paper/2204.02548/) (2022)
- [RobustART: Benchmarking Robustness on Architecture Design and Training Techniques](https://mlsanity.com/paper/2109.05211/) (2021)
- [When Does Contrastive Learning Preserve Adversarial Robustness from Pretraining to Finetuning?](https://mlsanity.com/paper/2111.01124/) (2021)
- [On the effectiveness of adversarial training against common corruptions](https://mlsanity.com/paper/2103.02325/) (2021)
- [Improved OOD Generalization via Adversarial Training and Pre-training](https://mlsanity.com/paper/2105.11144/) (2021)
- [Evaluating the Adversarial Robustness of Adaptive Test-time Defenses](https://mlsanity.com/paper/2202.13711/) (2022)
- [Relating Adversarially Robust Generalization to Flat Minima](https://mlsanity.com/paper/2104.04448/) (2021)
- [Understanding Robust Overfitting of Adversarial Training and Beyond](https://mlsanity.com/paper/2206.08675/) (2022)
- [Adversarial Robustness under Long-Tailed Distribution](https://mlsanity.com/paper/2104.02703/) (2021)
- [Online Adversarial Purification based on Self-Supervision](https://mlsanity.com/paper/2101.09387/) (2021)
- [Consistency Regularization for Adversarial Robustness](https://mlsanity.com/paper/2103.04623/) (2021)
- [Enhancing Adversarial Training with Second-Order Statistics of Weights](https://mlsanity.com/paper/2203.06020/) (2022)
- [LiBRe: A Practical Bayesian Approach to Adversarial Detection](https://mlsanity.com/paper/2103.14835/) (2021)
- [Understanding and Improving Robustness of Vision Transformers through Patch-based Negative Augmentation](https://mlsanity.com/paper/2110.07858/) (2021)
- [Biometrics: Trust, but Verify](https://mlsanity.com/paper/2105.06625/) (2021)
- [Adversarial Robustness with Semi-Infinite Constrained Learning](https://mlsanity.com/paper/2110.15767/) (2021)
- [Self-Ensemble Adversarial Training for Improved Robustness](https://mlsanity.com/paper/2203.09678/) (2022)
- [Probabilistically Robust Learning: Balancing Average- and Worst-case Performance](https://mlsanity.com/paper/2202.01136/) (2022)
- [Physical Adversarial Attacks on an Aerial Imagery Object Detector](https://mlsanity.com/paper/2108.11765/) (2021)
- [Prior-Guided Adversarial Initialization for Fast Adversarial Training](https://mlsanity.com/paper/2207.08859/) (2022)
- [Improving Fast Adversarial Training with Prior-Guided Knowledge](https://mlsanity.com/paper/2304.00202/) (2023)
- [PatchZero: Defending against Adversarial Patch Attacks by Detecting and Zeroing the Patch](https://mlsanity.com/paper/2207.01795/) (2022)

## Abstract (arXiv)

In this paper, we develop improved techniques for defending against adversarial examples at scale. First, we implement the state of the art version of adversarial training at unprecedented scale on ImageNet and investigate whether it remains effective in this setting - an important open scientific question (Athalye et al., 2018). Next, we introduce enhanced defenses using a technique we call logit pairing, a method that encourages logits for pairs of examples to be similar. When applied to clean examples and their adversarial counterparts, logit pairing improves accuracy on adversarial examples over vanilla adversarial training; we also find that logit pairing on clean examples only is competitive with adversarial training in terms of accuracy on two datasets. Finally, we show that adversarial logit pairing achieves the state of the art defense on ImageNet against PGD white box attacks, with an accuracy improvement from 1.5% to 27.9%. Adversarial logit pairing also successfully damages the current state of the art defense against black box attacks on ImageNet (Tramer et al., 2018), dropping its accuracy from 66.6% to 47.1%. With this new accuracy drop, adversarial logit pairing ties with Tramer et al.(2018) for the state of the art on black box attacks on ImageNet.