# Improving Adversarial Robustness via Probabilistically Compact Loss with Logit Constraints

arXiv: https://arxiv.org/abs/2012.07688 · HTML: https://mlsanity.com/paper/2012.07688/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Dec 2020.

## Built on by (most cited first, 10 of 10)

- [Hijacking Large Language Models via Adversarial In-Context Learning](https://mlsanity.com/paper/2311.09948/) (2023)
- [A.I. Robustness: a Human-Centered Perspective on Technological Challenges and Opportunities](https://mlsanity.com/paper/2210.08906/) (2022)
- [Negative Flux Aggregation to Estimate Feature Attributions](https://mlsanity.com/paper/2301.06989/) (2023)
- [Fairness-aware Vision Transformer via Debiased Self-Attention](https://mlsanity.com/paper/2301.13803/) (2023)
- [Improving Adversarial Robustness with Hypersphere Embedding and Angular-based Regularizations](https://mlsanity.com/paper/2303.08289/) (2023)
- [To Make Yourself Invisible with Adversarial Semantic Contours](https://mlsanity.com/paper/2303.00284/) (2023)
- [Saliency Guided Adversarial Training for Learning Generalizable Features with Applications to Medical Imaging Classification System](https://mlsanity.com/paper/2209.04326/) (2022)
- [Learning Compact Features via In-Training Representation Alignment](https://mlsanity.com/paper/2211.13332/) (2022)
- [Coupling User Preference with External Rewards to Enable Driver-centered and Resource-aware EV Charging Recommendation](https://mlsanity.com/paper/2210.12693/) (2022)
- [Improving Adversarial Robustness via Feature Pattern Consistency Constraint](https://mlsanity.com/paper/2406.08829/) (2024)

## Abstract (arXiv)

Convolutional neural networks (CNNs) have achieved state-of-the-art performance on various tasks in computer vision. However, recent studies demonstrate that these models are vulnerable to carefully crafted adversarial samples and suffer from a significant performance drop when predicting them. Many methods have been proposed to improve adversarial robustness (e.g., adversarial training and new loss functions to learn adversarially robust feature representations). Here we offer a unique insight into the predictive behavior of CNNs that they tend to misclassify adversarial samples into the most probable false classes. This inspires us to propose a new Probabilistically Compact (PC) loss with logit constraints which can be used as a drop-in replacement for cross-entropy (CE) loss to improve CNN's adversarial robustness. Specifically, PC loss enlarges the probability gaps between true class and false classes meanwhile the logit constraints prevent the gaps from being melted by a small perturbation. We extensively compare our method with the state-of-the-art using large scale datasets under both white-box and black-box attacks to demonstrate its effectiveness. The source codes are available from the following url: https://github.com/xinli0928/PC-LC.