# Few-Shot Calibration of Set Predictors via Meta-Learned Cross-Validation-Based Conformal Prediction

arXiv: https://arxiv.org/abs/2210.03067 · HTML: https://mlsanity.com/paper/2210.03067/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- Builds on 38 of its 67 references that are in the map.
- First posted on arXiv 6 Oct 2022.

## Builds on (most cited first, 30 of 38)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://mlsanity.com/paper/1912.01703/) (2019)
- [Very Deep Convolutional Networks for Large-Scale Image Recognition](https://mlsanity.com/paper/1409.1556/) (2014)
- [On Calibration of Modern Neural Networks](https://mlsanity.com/paper/1706.04599/) (2017)
- [Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks](https://mlsanity.com/paper/1703.03400/) (2017)
- [Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification](https://mlsanity.com/paper/1502.01852/) (2015)
- [Prototypical Networks for Few-shot Learning](https://mlsanity.com/paper/1703.05175/) (2017)
- [Matching Networks for One Shot Learning](https://mlsanity.com/paper/1606.04080/) (2016)
- [Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs)](https://mlsanity.com/paper/1511.07289/) (2015)
- [Conformalized Quantile Regression](https://mlsanity.com/paper/1905.03222/) (2019)
- [Conformal Prediction Under Covariate Shift](https://mlsanity.com/paper/1904.06019/) (2019)
- [Classification with Valid and Adaptive Coverage](https://mlsanity.com/paper/2006.02544/) (2020)
- [Probabilistic Model-Agnostic Meta-Learning](https://mlsanity.com/paper/1806.02817/) (2018)
- [Predictive inference with the jackknife+](https://mlsanity.com/paper/1905.02928/) (2019)
- [Bayesian Model-Agnostic Meta-Learning](https://mlsanity.com/paper/1806.03836/) (2018)
- [Recasting Gradient-Based Meta-Learning as Hierarchical Bayes](https://mlsanity.com/paper/1801.08930/) (2018)
- [A Primer on PAC-Bayesian Learning](https://mlsanity.com/paper/1901.05353/) (2019)
- [On the Convergence Theory of Gradient-Based Model-Agnostic Meta-Learning Algorithms](https://mlsanity.com/paper/1908.10400/) (2019)
- [Over the Air Deep Learning Based Radio Signal Classification](https://mlsanity.com/paper/1712.04578/) (2017)
- [Differentiable Ranks and Sorting using Optimal Transport](https://mlsanity.com/paper/1905.11885/) (2019)
- [Learning Optimal Conformal Classifiers](https://mlsanity.com/paper/2110.09192/) (2021)
- [Convolutional Radio Modulation Recognition Networks](https://mlsanity.com/paper/1602.04105/) (2016)
- [Meta-Learning without Memorization](https://mlsanity.com/paper/1912.03820/) (2019)
- [Meta-Learning by Adjusting Priors Based on Extended PAC-Bayes Theory](https://mlsanity.com/paper/1711.01244/) (2017)
- [CD-split and HPD-split: efficient conformal regions in high dimensions](https://mlsanity.com/paper/2007.12778/) (2020)
- [Training Uncertainty-Aware Classifiers with Conformalized Deep Learning](https://mlsanity.com/paper/2205.05878/) (2022)
- [Generalized Variational Inference: Three arguments for deriving new Posteriors](https://mlsanity.com/paper/1904.02063/) (2019)
- [Few-shot Conformal Prediction with Auxiliary Tasks](https://mlsanity.com/paper/2102.08898/) (2021)
- [Patterns of Scalable Bayesian Inference](https://mlsanity.com/paper/1602.05221/) (2016)
- [Locally Valid and Discriminative Prediction Intervals for Deep Learning Models](https://mlsanity.com/paper/2106.00225/) (2021)

## Built on by (most cited first, 17 of 17)

- [Semi-Supervised Risk Control via Prediction-Powered Inference](https://mlsanity.com/paper/2412.11174/) (2024)
- [Generalization and Informativeness of Conformal Prediction](https://mlsanity.com/paper/2401.11810/) (2024)
- [Synthetic-Powered Predictive Inference](https://mlsanity.com/paper/2505.13432/) (2025)
- [Knowing When to Stop: Delay-Adaptive Spiking Neural Network Classifiers with Reliability Guarantees](https://mlsanity.com/paper/2305.11322/) (2023)
- [Cross-Validation Conformal Risk Control](https://mlsanity.com/paper/2401.11974/) (2024)
- [Guaranteed Dynamic Scheduling of Ultra-Reliable Low-Latency Traffic via Conformal Prediction](https://mlsanity.com/paper/2302.07675/) (2023)
- [On the Temperature of Bayesian Graph Neural Networks for Conformal Prediction](https://mlsanity.com/paper/2310.11479/) (2023)
- [Dynamic Estimation Loss Control in Variational Quantum Sensing via Online Conformal Inference](https://mlsanity.com/paper/2505.23389/) (2025)
- [Calibration-Aware Bayesian Learning](https://mlsanity.com/paper/2305.07504/) (2023)
- [Semantic Meta-Split Learning: A TinyML Scheme for Few-Shot Wireless Image Classification](https://mlsanity.com/paper/2409.12978/) (2024)
- [Reliable Wireless Indoor Localization via Cross-Validated Prediction-Powered Calibration](https://mlsanity.com/paper/2507.20268/) (2025)
- [Optimizing In-Context Learning for Efficient Full Conformal Prediction](https://mlsanity.com/paper/2509.01840/) (2025)
- [CAOS: Conformal Aggregation of One-Shot Predictors](https://mlsanity.com/paper/2601.05219/) (2026)
- [Beyond Fixed False Discovery Rates: Post-Hoc Conformal Selection with E-Variables](https://mlsanity.com/paper/2604.11305/) (2026)
- [Conformal Changepoint Localization and Root Cause Analysis with Corrupted Observations](https://mlsanity.com/paper/2607.26481/) (2026)
- [Calibrating AI Models for Wireless Communications via Conformal Prediction](https://mlsanity.com/paper/2212.07775/) (2022)
- [Position: Artificial Intelligence Needs Meta Intelligence -- the Case for Metacognitive AI](https://mlsanity.com/paper/2605.15567/) (2026)

## Abstract (arXiv)

Conventional frequentist learning is known to yield poorly calibrated models that fail to reliably quantify the uncertainty of their decisions. Bayesian learning can improve calibration, but formal guarantees apply only under restrictive assumptions about correct model specification. Conformal prediction (CP) offers a general framework for the design of set predictors with calibration guarantees that hold regardless of the underlying data generation mechanism. However, when training data are limited, CP tends to produce large, and hence uninformative, predicted sets. This paper introduces a novel meta-learning solution that aims at reducing the set prediction size. Unlike prior work, the proposed meta-learning scheme, referred to as meta-XB, (i) builds on cross-validation-based CP, rather than the less efficient validation-based CP; and (ii) preserves formal per-task calibration guarantees, rather than less stringent task-marginal guarantees. Finally, meta-XB is extended to adaptive non-conformal scores, which are shown empirically to further enhance marginal per-input calibration.