Precision-Varying Prediction (PVP): Robustifying ASR systems against adversarial attacks

With the increasing deployment of automated and agentic systems, ensuring the adversarial robustness of automatic speech recognition (ASR) models has become highly relevant. We observe that changing the precision of an ASR model during inference reduces the likelihood of adversarial attacks to succeed. We take advantage of this fact to make models more robust simply by randomly sampling the precision during prediction. Moreover, this insight can be turned into an adversarial example detection strategy by implementing a simple Gaussian classifier that thresholds the differences between outputs of models run with different precision. To further enhance security boundaries, we combine the approach with an existing uncertainty-based defense mechanism, which forces adaptive adversaries to introduce highly perceptible noise to bypass detection. An experimental analysis across various ASR models, languages, and attack types demonstrates a significant increase in adversarial robustness, competitive detection capabilities, and resistance to adaptive threats.