# Confidence Calibration for Systems with Cascaded Predictive Modules

arXiv: https://arxiv.org/abs/2309.12510 · HTML: https://mlsanity.com/paper/2309.12510/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 21 of its 31 references that are in the map.
- First posted on arXiv 21 Sep 2023.

## Builds on (most cited first, 21 of 21)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification](https://mlsanity.com/paper/1502.01852/) (2015)
- [Matterport3D: Learning from RGB-D Data in Indoor Environments](https://mlsanity.com/paper/1709.06158/) (2017)
- [Distribution-Free Predictive Inference For Regression](https://mlsanity.com/paper/1604.04173/) (2016)
- [Conformalized Quantile Regression](https://mlsanity.com/paper/1905.03222/) (2019)
- [Conformal Prediction Under Covariate Shift](https://mlsanity.com/paper/1904.06019/) (2019)
- [Adaptive Conformal Inference Under Distribution Shift](https://mlsanity.com/paper/2106.00170/) (2021)
- [Learning to Recover 3D Scene Shape from a Single Image](https://mlsanity.com/paper/2012.09365/) (2020)
- [Conformal Inference for Online Prediction with Arbitrary Distribution Shifts](https://mlsanity.com/paper/2208.08401/) (2022)
- [Distribution-free uncertainty quantification for classification under label shift](https://mlsanity.com/paper/2103.03323/) (2021)
- [Evaluating Uncertainty Quantification in End-to-End Autonomous Driving Control](https://mlsanity.com/paper/1811.06817/) (2018)
- [Transferable Calibration with Lower Bias and Variance in Domain Adaptation](https://mlsanity.com/paper/2007.08259/) (2020)
- [Robust Calibration with Multi-domain Temperature Scaling](https://mlsanity.com/paper/2206.02757/) (2022)
- [Confidence Calibration for Domain Generalization under Covariate Shift](https://mlsanity.com/paper/2104.00742/) (2021)
- [Trusted AI in Multi-agent Systems: An Overview of Privacy and Security for Distributed Learning](https://mlsanity.com/paper/2202.09027/) (2022)
- [Reliable and Trustworthy Machine Learning for Health Using Dataset Shift Detection](https://mlsanity.com/paper/2110.14019/) (2021)
- [Unsupervised Calibration under Covariate Shift](https://mlsanity.com/paper/2006.16405/) (2020)
- [Modular Conformal Calibration](https://mlsanity.com/paper/2206.11468/) (2022)
- [Pose Guided Structured Region Ensemble Network for Cascaded Hand Pose Estimation](https://mlsanity.com/paper/1708.03416/) (2017)
- [Hidden Heterogeneity: When to Choose Similarity-Based Calibration](https://mlsanity.com/paper/2202.01840/) (2022)

## Built on by (most cited first, 2 of 2)

- [Metric-Guided Conformal Bounds for Probabilistic Image Reconstruction](https://mlsanity.com/paper/2404.15274/) (2024)
- [Decomposition-Based Modular Conformal Prediction for Two-Stage Modeling](https://mlsanity.com/paper/2510.04406/) (2025)

## Abstract (arXiv)

Existing conformal prediction algorithms estimate prediction intervals at target confidence levels to characterize the performance of a regression model on new test samples. However, considering an autonomous system consisting of multiple modules, prediction intervals constructed for individual modules fall short of accommodating uncertainty propagation over different modules and thus cannot provide reliable predictions on system behavior. We address this limitation and present novel solutions based on conformal prediction to provide prediction intervals calibrated for a predictive system consisting of cascaded modules (e.g., an upstream feature extraction module and a downstream regression module). Our key idea is to leverage module-level validation data to characterize the system-level error distribution without direct access to end-to-end validation data. We provide theoretical justification and empirical experimental results to demonstrate the effectiveness of proposed solutions. In comparison to prediction intervals calibrated for individual modules, our solutions generate improved intervals with more accurate performance guarantees for system predictions, which are demonstrated on both synthetic systems and real-world systems performing overlap prediction for indoor navigation using the Matterport3D dataset.