# Upper Bound of Bayesian Generalization Error in Partial Concept Bottleneck Model (CBM): Partial CBM outperforms naive CBM

arXiv: https://arxiv.org/abs/2403.09206 · HTML: https://mlsanity.com/paper/2403.09206/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 27 of its 75 references that are in the map.
- First posted on arXiv 14 Mar 2024.

## Builds on (most cited first, 27 of 27)

- [How transferable are features in deep neural networks?](https://mlsanity.com/paper/1411.1792/) (2014)
- [Concept Bottleneck Models](https://mlsanity.com/paper/2007.04612/) (2020)
- [Towards Robust Interpretability with Self-Explaining Neural Networks](https://mlsanity.com/paper/1806.07538/) (2018)
- [Stochastic Gradient Descent as Approximate Bayesian Inference](https://mlsanity.com/paper/1704.04289/) (2017)
- [Acquisition of Chess Knowledge in AlphaZero](https://mlsanity.com/paper/2111.09259/) (2021)
- [Ethical Machine Learning in Health Care](https://mlsanity.com/paper/2009.10576/) (2020)
- [Explainable Object-induced Action Decision for Autonomous Vehicles](https://mlsanity.com/paper/2003.09405/) (2020)
- [Concept Bottleneck Model with Additional Unsupervised Concepts](https://mlsanity.com/paper/2202.01459/) (2022)
- [Deep Learning is Singular, and That's Good](https://mlsanity.com/paper/2010.11560/) (2020)
- [Static and Dynamic Concepts for Self-supervised Video Representation Learning](https://mlsanity.com/paper/2207.12795/) (2022)
- [A Bayesian information criterion for singular models](https://mlsanity.com/paper/1309.0911/) (2013)
- [The Exact Asymptotic Form of Bayesian Generalization Error in Latent Dirichlet Allocation](https://mlsanity.com/paper/2008.01304/) (2020)
- [Fractional Langevin Monte Carlo: Exploring L'evy Driven Stochastic Differential Equations for Markov Chain Monte Carlo](https://mlsanity.com/paper/1706.03649/) (2017)
- [Marginal likelihood and model selection for Gaussian latent tree and forest models](https://mlsanity.com/paper/1412.8285/) (2014)
- [Upper Bound of Bayesian Generalization Error in Non-negative Matrix Factorization](https://mlsanity.com/paper/1612.04112/) (2016)
- [Estimating Real Log Canonical Thresholds](https://mlsanity.com/paper/1906.01341/) (2019)
- [Asymptotic Behavior of Bayesian Generalization Error in Multinomial Mixtures](https://mlsanity.com/paper/2203.06884/) (2022)
- [Learning to Predict with Supporting Evidence: Applications to Clinical Risk Prediction](https://mlsanity.com/paper/2103.02768/) (2021)
- [Estimating the Local Learning Coefficient at Scale](https://mlsanity.com/paper/2402.03698/) (2024)
- [Variational Approximation Error in Bayesian Non-negative Matrix Factorization](https://mlsanity.com/paper/1809.02963/) (2018)
- [Mathematical Theory of Bayesian Statistics for Unknown Information Source](https://mlsanity.com/paper/2206.05630/) (2022)
- [Bayesian Free Energy of Deep ReLU Neural Network in Overparametrized Cases](https://mlsanity.com/paper/2303.15739/) (2023)
- [Asymptotic Bayesian Generalization Error in Latent Dirichlet Allocation and Stochastic Matrix Factorization](https://mlsanity.com/paper/1709.04212/) (2017)
- [Stochastic natural gradient descent draws posterior samples in function space](https://mlsanity.com/paper/1806.09597/) (2018)
- [Free energy of Bayesian Convolutional Neural Network with Skip Connection](https://mlsanity.com/paper/2307.01417/) (2023)
- [Bayesian Generalization Error in Linear Neural Networks with Concept Bottleneck Structure and Multitask Formulation](https://mlsanity.com/paper/2303.09154/) (2023)
- [Clinical outcome prediction under hypothetical interventions -- a representation learning framework for counterfactual reasoning](https://mlsanity.com/paper/2205.07234/) (2022)

## Built on by (most cited first, 1 of 1)

- [Towards Reasonable Concept Bottleneck Models](https://mlsanity.com/paper/2506.05014/) (2025)

## Abstract (arXiv)

Concept Bottleneck Model (CBM) is a methods for explaining neural networks. In CBM, concepts which correspond to reasons of outputs are inserted in the last intermediate layer as observed values. It is expected that we can interpret the relationship between the output and concept similar to linear regression. However, this interpretation requires observing all concepts and decreases the generalization performance of neural networks. Partial CBM (PCBM), which uses partially observed concepts, has been devised to resolve these difficulties. Although some numerical experiments suggest that the generalization performance of PCBMs is almost as high as that of the original neural networks, the theoretical behavior of its generalization error has not been yet clarified since PCBM is singular statistical model. In this paper, we reveal the Bayesian generalization error in PCBM with a three-layered and linear architecture. The result indcates that the structure of partially observed concepts decreases the Bayesian generalization error compared with that of CBM (full-observed concepts).