mlsanityData to 7 Oct 2026

Bayesian Neural Networks Avoid Encoding Complex and Perturbation-Sensitive Concepts

arXiv 2302.13095 ↗First posted 25 Feb 2023cs.LG cs.AI cs.CVcited by 9

Not on the map. Why?

Abstract (arXiv)

In this paper, we focus on mean-field variational Bayesian Neural Networks (BNNs) and explore the representation capacity of such BNNs by investigating which types of concepts are less likely to be encoded by the BNN. It has been observed and studied that a relatively small set of interactive concepts usually emerge in the knowledge representation of a sufficiently-trained neural network, and such concepts can faithfully explain the network output. Based on this, our study proves that compared to standard deep neural networks (DNNs), it is less likely for BNNs to encode complex concepts. Experiments verify our theoretical proofs. Note that the tendency to encode less complex concepts does not necessarily imply weak representation power, considering that complex concepts exhibit low generalization power and high adversarial vulnerability. The code is available at https://github.com/sjtu-xai-lab/BNN-concepts.

Key facts. Cited by 9 papers in the map (data through 7 Oct 2026). Builds on 23 of its 43 references that are in the map. First posted on arXiv 25 Feb 2023.

Builds on

23 of 43 references.

References · 23

  1. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  2. 2017Towards Deep Learning Models Resistant to Adversarial Attackscited by 6,326
  3. 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,416
  4. 2018Neural Network Acceptability Judgmentscited by 952
  5. 2019Adversarial Examples Are Not Bugs, They Are Featurescited by 950
  6. 2021ResMLP: Feedforward networks for image classification with data-efficient trainingcited by 338
  7. 2018Consistent Individualized Feature Attribution for Tree Ensemblescited by 279
  8. 2020Being Bayesian, Even Just a Bit, Fixes Overconfidence in ReLU Networkscited by 227
  9. 2019The Shapley Taylor Interaction Indexcited by 143
  10. 2021Bayesian Neural Network Priors Revisitedcited by 121
  11. 2019Explaining Deep Neural Networks with a Polynomial Time Algorithm for Shapley Values Approximationcited by 115
  12. 2020Explaining Explanations: Axiomatic Feature Interactions for Deep Networkscited by 103
11 more
  1. 2019On the Connection Between Adversarial Robustness and Saliency Map Interpretabilitycited by 80
  2. 2020A Unified Approach to Interpreting and Boosting Adversarial Transferabilitycited by 71
  3. 2020Robustness of Bayesian Neural Networks to Gradient-Based Attackscited by 48
  4. 2021Discovering and Explaining the Representation Bottleneck of DNNscited by 42
  5. 2021Defining and Quantifying the Emergence of Sparse Concepts in DNNscited by 36
  6. 2020Interpreting and Boosting Dropout from a Game-Theoretic Viewcited by 34
  7. 2019Specifying Weight Priors in Bayesian Deep Neural Networks with Empirical Bayescited by 32
  8. 2021A Unified Game-Theoretic Interpretation of Adversarial Robustnesscited by 17
  9. 2021Can We Faithfully Represent Masked States to Compute Shapley Values on a DNN?cited by 10
  10. 2018Sufficient Conditions for Idealised Models to Have No Adversarial Examples: a Theoretical and Empirical Study with Bayesian Neural Networkscited by 9
  11. 2020Dropout as a Regularizer of Interaction Effectscited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 3; 2025: 2; 2026: 4.

9 papers.

  1. 2024Explainable and Interpretable Multimodal Large Language Models: A Comprehensive Surveycited by 52
  2. 2024Why pre-training is beneficial for downstream classification tasks?cited by 1
  3. 2025Technical Report: Quantifying and Analyzing the Generalization Power of a DNNcited by 1
  4. 2026Cross-LLM Consistency in Inference: Evidence from Shared Interactionscited by 1
  5. 2024Disentangling Regional Primitives for Image Generationcited by 0
  6. 2025S2CFormer: Revisiting the RD-Latency Trade-off in Transformer-based Learned Image Compressioncited by 0
  7. 2026Reconciling Contradictory Views on the Effectiveness of SFT in LLMs: An Interaction Perspectivecited by 0
  8. 2026A Unified Approach to Interpreting Knowledge Distillation for Large Language Models via Interactionscited by 0
  9. 2026Evaluating and Explaining Prompt Sensitivity of LLMs Using Interactionscited by 0