Bayesian Conformal Prediction as a Decision Risk Problem

We propose Bayesian Conformal Prediction (BCP), a framework that combines Bayesian posterior predictive distributions with PAC-style conformal risk control to produce prediction sets with finite-sample coverage guarantees. Standard quantile-threshold conformal methods often construct prediction sets using a single fixed threshold, which typically yields connected prediction sets. While valid, such sets can be inefficient when the posterior predictive distribution is multimodal, since they may span low-density regions between separated modes. The main contribution of BCP is to formulate conformal prediction as a decision-risk optimisation problem, extending standard fixed quantile-threshold sets to optimised highest posterior density (HPD) prediction sets. These sets can be disjoint, concentrating probability mass on separated high-density regions. Validity is enforced using a PAC-style risk constraint, which provides coverage control even when the Bayesian model is misspecified. In standard nested-threshold settings, BCP recovers the smallest feasible threshold, aligning with existing PAC-based approaches. In the multimodal experiment, HPD geometry substantially improves efficiency, reducing mean prediction set size from 4.82 to 2.07 while satisfying the target PAC pass rate. Across regression, classification, and distribution-shift experiments, BCP maintains reliable coverage under model misspecification, whereas Bayesian credible intervals can fail to preserve nominal coverage.