Probabilistic Adversarial Training

Building on a probabilistic perspective in which adversarial examples arise from the overlap between a distance-based distribution pdis and a victim-classifier-induced distribution pvic, we start from a simple intuition: adversarial examples become harder to generate when these two distributions are pushed apart, as their overlap becomes smaller, thereby increasing robustness. This intuition naturally motivates a KL-based robustness objective. We then prove that KL(pdis|pvic)-log Zvic is a lower bound on probabilistic robustness (PR), where Zvic denotes the normalizing constant of pvic. Since PR is generally intractable to compute directly, maximizing this KL-based lower bound provides a tractable surrogate objective for improving PR. We further show that this objective recovers a scaled form of adversarial training, offering a probabilistic interpretation of adversarial training and a principled route to robustness improvement. We call the resulting method probabilistic adversarial training. Experiments show that it consistently improves PR, and ablation studies demonstrate that the induced scaling factor can even enhance the PR of non-probabilistic adversarial training methods.