mlsanityData to 7 Oct 2026

Revisiting Hilbert-Schmidt Information Bottleneck for Adversarial Robustness

arXiv 2106.02734 ↗First posted 4 Jun 2021cs.LGcited by 20

Not on the map. Why?

Abstract (arXiv)

We investigate the HSIC (Hilbert-Schmidt independence criterion) bottleneck as a regularizer for learning an adversarially robust deep neural network classifier. In addition to the usual cross-entropy loss, we add regularization terms for every intermediate layer to ensure that the latent representations retain useful information for output prediction while reducing redundant information. We show that the HSIC bottleneck enhances robustness to adversarial attacks both theoretically and experimentally. In particular, we prove that the HSIC bottleneck regularizer reduces the sensitivity of the classifier to adversarial examples. Our experiments on multiple benchmark datasets and architectures demonstrate that incorporating an HSIC bottleneck regularizer attains competitive natural accuracy and improves adversarial robustness, both with and without adversarial examples during training. Our code and adversarially robust models are publicly available.

Key facts. Cited by 20 papers in the map (data through 7 Oct 2026). Builds on 27 of its 44 references that are in the map. First posted on arXiv 4 Jun 2021.

Builds on

27 of 44 references.

References · 27

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2014Explaining and Harnessing Adversarial Examplescited by 7,287
  3. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  4. 2013Intriguing properties of neural networkscited by 5,114
  5. 2013Auto-Encoding Variational Bayescited by 3,980
  6. 2016Wide Residual Networkscited by 3,057
  7. 2016Towards Evaluating the Robustness of Neural Networkscited by 2,976
  8. 2019Theoretically Principled Trade-off between Robustness and Accuracycited by 1,510
  9. 2020Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-free attackscited by 1,442
  10. 2015DeepFool: a simple and accurate method to fool deep neural networkscited by 1,315
  11. 2016Adversarial Machine Learning at Scalecited by 934
  12. 2016Deep Variational Information Bottleneckcited by 915
15 more
  1. 2016Practical Black-Box Attacks against Machine Learningcited by 893
  2. 2018Robustness May Be at Odds with Accuracycited by 873
  3. 2015Deep Learning and the Information Bottleneck Principlecited by 793
  4. 2017Opening the Black Box of Deep Neural Networks via Informationcited by 494
  5. 2019Fooling automated surveillance cameras: adversarial patches to attack person detectioncited by 268
  6. 2020Understanding and Mitigating the Tradeoff Between Robustness and Accuracycited by 148
  7. 2019Adversarial Robustness vs Model Compression, or Both?cited by 98
  8. 2019The HSIC Bottleneck: Deep Learning without Back-Propagationcited by 77
  9. 2020The Conditional Entropy Bottleneckcited by 75
  10. 2018Deep Defense: Training DNNs with Improved Adversarial Robustnesscited by 41
  11. 2019Robust Learning with the Hilbert-Schmidt Independence Criterioncited by 36
  12. 2019Are Self-Driving Cars Secure? Evasion Attacks against Deep Neural Networks for Steering Angle Predictioncited by 16
  13. 2020Open-World Class Discovery with Kernel Networkscited by 12
  14. 2019Deep Kernel Learning for Clusteringcited by 3
  15. 2019Solving Interpretable Kernel Dimension Reductioncited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 2; 2023: 6; 2024: 5; 2025: 4; 2026: 3.

20 papers.

Sort:

By area:

  1. 2023DualHSIC: HSIC-Bottleneck and Alignment for Continual Learningcited by 15
  2. 2023A Unified Framework of Graph Information Bottleneck for Robustness and Membership Privacycited by 14
  3. 2022Pruning Adversarially Robust Neural Networks without Adversarial Examplescited by 12
  4. 2023MIMIR: Masked Image Modeling for Mutual Information-based Adversarial Robustnesscited by 12
  5. 2024Towards Continual Learning Desiderata via HSIC-Bottleneck Orthogonalization and Equiangular Embeddingcited by 9
  6. 2024Cauchy-Schwarz Divergence Information Bottleneck for Regressioncited by 9
  7. 2024End-to-End Training Induces Information Bottleneck through Layer-Role Differentiation: A Comparative Analysis with Layer-wise Trainingcited by 8
  8. 2024DD-RobustBench: An Adversarial Robustness Benchmark for Dataset Distillationcited by 8
  9. 2026ActQuant: Sub-4-bit Action-Guided Quantization for Vision-Language-Action Modelscited by 7
  10. 2022A Comprehensive Review of Trends, Applications and Challenges In Out-of-Distribution Detectioncited by 5
  11. 2025Visual Instruction Bottleneck Tuningcited by 5
  12. 2023Exploring Robust Features for Improving Adversarial Robustnesscited by 4
8 more
  1. 2025Explaining Grokking and Information Bottleneck through Neural Collapse Emergencecited by 4
  2. 2025A Generalized Information Bottleneck Theory of Deep Learningcited by 4
  3. 2023IB-RAR: Information Bottleneck as Regularizer for Adversarial Robustnesscited by 2
  4. 2024Exploring Heterogeneity and Uncertainty for Graph-based Cognitive Diagnosis Models in Intelligent Educationcited by 2
  5. 2025H-SPLID: HSIC-based Saliency Preserving Latent Information Decompositioncited by 2
  6. 2023Adversarial Attacks to Latent Representations of Distributed Neural Networks in Split Computingcited by 1
  7. 2026InfoTok: Information-Theoretic Regularization for Capacity-Constrained Shared Visual Tokenization in Unified MLLMscited by 0
  8. 2026Improving the Robustness/Accuracy Tradeoff Against Adversarial Attacks Using Information Bottleneck Distillation Through Dual Teacherscited by 0