Not Just How Much, But Where: Decomposing Epistemic Uncertainty into Per-Class Contributions

In safety-critical classification, the cost of failure is often asymmetric, yet Bayesian deep learning summarises epistemic uncertainty with a single scalar, mutual information (MI), that cannot distinguish whether a model's ignorance involves a benign or safety-critical class. We decompose MI into a per-class vector Ck(x)=σk2/(2μk), with μk=mathbbE[pk] and σk2=Var[pk] across posterior samples. The decomposition follows from a second-order Taylor expansion of the entropy; the 1/μk weighting corrects boundary suppression and makes Ck comparable across rare and common classes. By construction sumk Ck ≈ MI, and a companion skewness diagnostic flags inputs where the approximation degrades. After characterising the axiomatic properties of Ck, we validate it on three tasks: (i) selective prediction for diabetic retinopathy, where critical-class Ck reduces selective risk by 34.7% over MI and 56.2% over variance baselines; (ii) out-of-distribution detection on clinical and image benchmarks, where sumk Ck achieves the highest AUROC and the per-class view exposes asymmetric shifts invisible to MI; and (iii) a controlled label-noise study in which sumk Ck shows less sensitivity to injected aleatoric noise than MI under end-to-end Bayesian training, while both metrics degrade under transfer learning. Across all tasks, the quality of the posterior approximation shapes uncertainty at least as strongly as the choice of metric, suggesting that how uncertainty is propagated through the network matters as much as how it is measured.