Bridging Models to Defend: A Population-Based Strategy for Robust Adversarial Defense

Adversarial robustness is a critical measure of a neural network's ability to withstand adversarial attacks at inference time. While robust training techniques have improved defenses against individual ℓp-norm attacks (e.g., ℓ2 or ℓ_∞), models remain vulnerable to diversified ℓp perturbations. To address this challenge, we propose a novel Robust Mode Connectivity (RMC)-oriented adversarial defense framework comprising two population-based learning phases. In Phase I, RMC searches the parameter space between two pre-trained models to construct a continuous path containing models with high robustness against multiple ℓp attacks. To improve efficiency, we introduce a Self-Robust Mode Connectivity (SRMC) module that accelerates endpoint generation in RMC. Building on RMC, Phase II presents RMC-based optimization, where RMC modules are composed to further enhance diversified robustness. To increase Phase II efficiency, we propose Efficient Robust Mode Connectivity (ERMC), which leverages ℓ1- and ℓ_∞-adversarially trained models to achieve robustness across a broad range of p-norms. An ensemble strategy is employed to further boost ERMC's performance. Extensive experiments across diverse datasets and architectures demonstrate that our methods significantly improve robustness against ℓ_∞, ℓ2, ℓ1, and hybrid attacks. Code is available at https://github.com/wangren09/MCGR.