# Federated Conformal Predictors for Distributed Uncertainty Quantification

arXiv: https://arxiv.org/abs/2305.17564 · HTML: https://mlsanity.com/paper/2305.17564/

## Key facts

- Cited by 46 papers in the map (data through 14 Oct 2026).
- Builds on 39 of its 64 references that are in the map.
- First posted on arXiv 27 May 2023.

## Builds on (most cited first, 30 of 39)

- [Communication-Efficient Learning of Deep Networks from Decentralized Data](https://mlsanity.com/paper/1602.05629/) (2016)
- [Scaling Laws for Neural Language Models](https://mlsanity.com/paper/2001.08361/) (2020)
- [EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks](https://mlsanity.com/paper/1905.11946/) (2019)
- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [Federated Optimization in Heterogeneous Networks](https://mlsanity.com/paper/1812.06127/) (2018)
- [Can You Trust Your Model's Uncertainty? Evaluating Predictive Uncertainty Under Dataset Shift](https://mlsanity.com/paper/1906.02530/) (2019)
- [On the Convergence of FedAvg on Non-IID Data](https://mlsanity.com/paper/1907.02189/) (2019)
- [Federated Learning with Non-IID Data](https://mlsanity.com/paper/1806.00582/) (2018)
- [Tackling the Objective Inconsistency Problem in Heterogeneous Federated Optimization](https://mlsanity.com/paper/2007.07481/) (2020)
- [Measuring the Effects of Non-Identical Data Distribution for Federated Visual Classification](https://mlsanity.com/paper/1909.06335/) (2019)
- [A Gentle Introduction to Conformal Prediction and Distribution-Free Uncertainty Quantification](https://mlsanity.com/paper/2107.07511/) (2021)
- [MedMNIST v2 -- A large-scale lightweight benchmark for 2D and 3D biomedical image classification](https://mlsanity.com/paper/2110.14795/) (2021)
- [Distribution-Free Predictive Inference For Regression](https://mlsanity.com/paper/1604.04173/) (2016)
- [The Future of Digital Health with Federated Learning](https://mlsanity.com/paper/2003.08119/) (2020)
- [Federated Learning on Non-IID Data Silos: An Experimental Study](https://mlsanity.com/paper/2102.02079/) (2021)
- [Agnostic Federated Learning](https://mlsanity.com/paper/1902.00146/) (2019)
- [Classification with Valid and Adaptive Coverage](https://mlsanity.com/paper/2006.02544/) (2020)
- [Uncertainty Sets for Image Classifiers using Conformal Prediction](https://mlsanity.com/paper/2009.14193/) (2020)
- [Federated Learning on Non-IID Data: A Survey](https://mlsanity.com/paper/2106.06843/) (2021)
- [The Non-IID Data Quagmire of Decentralized Machine Learning](https://mlsanity.com/paper/1910.00189/) (2019)
- [Distribution-Free, Risk-Controlling Prediction Sets](https://mlsanity.com/paper/2101.02703/) (2021)
- [Least Ambiguous Set-Valued Classifiers with Bounded Error Levels](https://mlsanity.com/paper/1609.00451/) (2016)
- [Learn then Test: Calibrating Predictive Algorithms to Achieve Risk Control](https://mlsanity.com/paper/2110.01052/) (2021)
- [Evaluating Deep Neural Networks Trained on Clinical Images in Dermatology with the Fitzpatrick 17k Dataset](https://mlsanity.com/paper/2104.09957/) (2021)
- [Revisiting Neural Scaling Laws in Language and Vision](https://mlsanity.com/paper/2209.06640/) (2022)
- [Uncertainty as a Form of Transparency: Measuring, Communicating, and Using Uncertainty](https://mlsanity.com/paper/2011.07586/) (2020)
- [FLamby: Datasets and Benchmarks for Cross-Silo Federated Learning in Realistic Healthcare Settings](https://mlsanity.com/paper/2210.04620/) (2022)
- [Image-to-Image Regression with Distribution-Free Uncertainty Quantification and Applications in Imaging](https://mlsanity.com/paper/2202.05265/) (2022)
- [Fair Conformal Predictors for Applications in Medical Imaging](https://mlsanity.com/paper/2109.04392/) (2021)

## Built on by (most cited first, 30 of 46)

- [Benchmarking LLMs via Uncertainty Quantification](https://mlsanity.com/paper/2401.12794/) (2024)
- [Conformal Prediction for Deep Classifier via Label Ranking](https://mlsanity.com/paper/2310.06430/) (2023)
- [Conformal Prediction: A Data Perspective](https://mlsanity.com/paper/2410.06494/) (2024)
- [Wasserstein-regularized Conformal Prediction under General Distribution Shift](https://mlsanity.com/paper/2501.13430/) (2025)
- [On Temperature Scaling and Conformal Prediction of Deep Classifiers](https://mlsanity.com/paper/2402.05806/) (2024)
- [Does confidence calibration improve conformal prediction?](https://mlsanity.com/paper/2402.04344/) (2024)
- [Multi-Source Conformal Inference Under Distribution Shift](https://mlsanity.com/paper/2405.09331/) (2024)
- [Federated Inference with Reliable Uncertainty Quantification over Wireless Channels via Conformal Prediction](https://mlsanity.com/paper/2308.04237/) (2023)
- [Efficient Conformal Prediction under Data Heterogeneity](https://mlsanity.com/paper/2312.15799/) (2023)
- [Privacy Preserving Federated Learning in Medical Imaging with Uncertainty Estimation](https://mlsanity.com/paper/2406.12815/) (2024)
- [Certifiably Byzantine-Robust Federated Conformal Prediction](https://mlsanity.com/paper/2406.01960/) (2024)
- [Investigating the Quality of DermaMNIST and Fitzpatrick17k Dermatological Image Datasets](https://mlsanity.com/paper/2401.14497/) (2024)
- [What If We Had Used a Different App? Reliable Counterfactual KPI Analysis in Wireless Systems](https://mlsanity.com/paper/2410.00150/) (2024)
- [Uncertainty quantification in metric spaces](https://mlsanity.com/paper/2405.05110/) (2024)
- [Distributed Conformal Prediction via Message Passing](https://mlsanity.com/paper/2501.14544/) (2025)
- [Efficient Federated Conformal Prediction with Group-Conditional Guarantee](https://mlsanity.com/paper/2603.14198/) (2026)
- [Marginal and training-conditional guarantees in one-shot federated conformal prediction](https://mlsanity.com/paper/2405.12567/) (2024)
- [Don't Reach for the Stars: Rethinking Topology for Resilient Federated Learning](https://mlsanity.com/paper/2508.05224/) (2025)
- [FedCF: Fair Federated Conformal Prediction](https://mlsanity.com/paper/2509.22907/) (2025)
- [Future-Proofing Medical Imaging with Privacy-Preserving Federated Learning and Uncertainty Quantification: A Review](https://mlsanity.com/paper/2409.16340/) (2024)
- [Conformal Prediction Adaptive to Unknown Subpopulation Shifts](https://mlsanity.com/paper/2506.05583/) (2025)
- [Conformal and kNN Predictive Uncertainty Quantification Algorithms in Metric Spaces](https://mlsanity.com/paper/2507.15741/) (2025)
- [Federated Conditional Conformal Prediction via Generative Models](https://mlsanity.com/paper/2510.13297/) (2025)
- [Multi-Distribution Robust Conformal Prediction](https://mlsanity.com/paper/2601.02998/) (2026)
- [Conformalized Neural Networks for Federated Uncertainty Quantification under Dual Heterogeneity](https://mlsanity.com/paper/2602.23296/) (2026)
- [Certified Adaptive Refresh: Anytime-Valid Monitoring for Federated Conformal RAG](https://mlsanity.com/paper/2605.29139/) (2026)
- [TPFL: A Trustworthy Personalized Federated Learning Framework via Subjective Logic](https://mlsanity.com/paper/2410.12316/) (2024)
- [Segmentation-Guided CT Synthesis with Pixel-Wise Conformal Uncertainty Bounds](https://mlsanity.com/paper/2503.08515/) (2025)
- [Partial Model Sharing Improves Byzantine Resilience in Federated Conformal Prediction](https://mlsanity.com/paper/2605.11684/) (2026)
- [Online Conformal Prediction: Enforcing monotonicity via Online Optimization](https://mlsanity.com/paper/2605.12668/) (2026)

## Abstract (arXiv)

Conformal prediction is emerging as a popular paradigm for providing rigorous uncertainty quantification in machine learning since it can be easily applied as a post-processing step to already trained models. In this paper, we extend conformal prediction to the federated learning setting. The main challenge we face is data heterogeneity across the clients - this violates the fundamental tenet of exchangeability required for conformal prediction. We propose a weaker notion of partial exchangeability, better suited to the FL setting, and use it to develop the Federated Conformal Prediction (FCP) framework. We show FCP enjoys rigorous theoretical guarantees and excellent empirical performance on several computer vision and medical imaging datasets. Our results demonstrate a practical approach to incorporating meaningful uncertainty quantification in distributed and heterogeneous environments. We provide code used in our experiments https://github.com/clu5/federated-conformal.