mlsanityData to 7 Oct 2026

Few-shot Conformal Prediction with Auxiliary Tasks

arXiv 2102.08898 ↗First posted 17 Feb 2021cs.LG cs.AI cs.CLcited by 53

Not on the map. Why?

Abstract (arXiv)

We develop a novel approach to conformal prediction when the target task has limited data available for training. Conformal prediction identifies a small set of promising output candidates in place of a single prediction, with guarantees that the set contains the correct answer with high probability. When training data is limited, however, the predicted set can easily become unusably large. In this work, we obtain substantially tighter prediction sets while maintaining desirable marginal guarantees by casting conformal prediction as a meta-learning paradigm over exchangeable collections of auxiliary tasks. Our conformalization algorithm is simple, fast, and agnostic to the choice of underlying model, learning algorithm, or dataset. We demonstrate the effectiveness of this approach across a number of few-shot classification and regression tasks in natural language processing, computer vision, and computational chemistry for drug discovery.

Key facts. Cited by 53 papers in the map (data through 7 Oct 2026). Builds on 28 of its 48 references that are in the map. First posted on arXiv 17 Feb 2021.

Builds on

28 of 48 references.

References · 28

  1. 2017Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networkscited by 4,861
  2. 2017On Calibration of Modern Neural Networkscited by 4,860
  3. 2015Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learningcited by 4,416
  4. 2016Simple and Scalable Predictive Uncertainty Estimation using Deep Ensemblescited by 3,888
  5. 2017Prototypical Networks for Few-shot Learningcited by 3,001
  6. 2016Matching Networks for One Shot Learningcited by 2,246
  7. 2016Concrete Problems in AI Safetycited by 1,582
  8. 2017Deep Setscited by 1,394
  9. 2016Distribution-Free Predictive Inference For Regressioncited by 657
  10. 2019Conformalized Quantile Regressioncited by 620
  11. 2019Conformal Prediction Under Covariate Shiftcited by 511
  12. 2019Generalizing from a Few Examples: A Survey on Few-Shot Learningcited by 462
16 more
  1. 2018Meta-learning with differentiable closed-form solverscited by 395
  2. 2017Training Confidence-calibrated Classifiers for Detecting Out-of-Distribution Samplescited by 389
  3. 2019Analyzing Learned Molecular Representations for Property Predictioncited by 373
  4. 2020Uncertainty Sets for Image Classifiers using Conformal Predictioncited by 365
  5. 2015Probabilistic Backpropagation for Scalable Learning of Bayesian Neural Networkscited by 295
  6. 2021Distribution-Free, Risk-Controlling Prediction Setscited by 278
  7. 2018FewRel: A Large-Scale Supervised Few-Shot Relation Classification Dataset with State-of-the-Art Evaluationcited by 254
  8. 2020Pitfalls of In-Domain Uncertainty Estimation and Ensembling in Deep Learningcited by 208
  9. 2018To Trust Or Not To Trust A Classifiercited by 205
  10. 2016Towards a Neural Statisticiancited by 119
  11. 2019Distributional conformal predictioncited by 104
  12. 2020Uncertainty Quantification Using Neural Networks for Molecular Property Predictioncited by 66
  13. 2019Few-shot Text Classification with Distributional Signaturescited by 58
  14. 2019Adaptive, Distribution-Free Prediction Intervals for Deep Networkscited by 56
  15. 2019With Malice Towards None: Assessing Uncertainty via Equalized Coveragecited by 43
  16. 2020Knowing what you know: valid and validated confidence sets in multiclass and multilabel predictioncited by 9

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 7; 2022: 12; 2023: 13; 2024: 8; 2025: 10; 2026: 3.

53 papers.

Sort:

By area:

  1. 2021A Gentle Introduction to Conformal Prediction and Distribution-Free Uncertainty Quantificationcited by 777
  2. 2021Learn then Test: Calibrating Predictive Algorithms to Achieve Risk Controlcited by 256
  3. 2023Conformal Language Modelingcited by 163
  4. 2021Consistent Accelerated Inference via Confident Adaptive Transformerscited by 73
  5. 2023Conformal Prediction for Deep Classifier via Label Rankingcited by 52
  6. 2024Conformal Prediction for Natural Language Processing: A Surveycited by 52
  7. 2023Federated Conformal Predictors for Distributed Uncertainty Quantificationcited by 46
  8. 2021PAC Prediction Sets Under Covariate Shiftcited by 42
  9. 2022Conformal Prediction Sets with Limited False Positivescited by 28
  10. 2022Conformal Prediction with Temporal Quantile Adjustmentscited by 25
  11. 2024Conformal Prediction: A Data Perspectivecited by 25
  12. 2021Conformal prediction for text infilling and part-of-speech predictioncited by 22
38 more
  1. 2022Semantic uncertainty intervals for disentangled latent spacescited by 22
  2. 2024Non-Exchangeable Conformal Language Generation with Nearest Neighborscited by 21
  3. 2025Multivariate Conformal Prediction using Optimal Transportcited by 21
  4. 2022Split Localized Conformal Predictioncited by 20
  5. 2021Stable Conformal Prediction Setscited by 19
  6. 2024Conformal Prediction for Class-wise Coverage via Augmented Label Rank Calibrationcited by 18
  7. 2022Few-Shot Calibration of Set Predictors via Meta-Learned Cross-Validation-Based Conformal Predictioncited by 17
  8. 2022PAC Prediction Sets for Meta-Learningcited by 16
  9. 2023Conformal Autoregressive Generation: Beam Search with Coverage Guaranteescited by 16
  10. 2023Selective Generation for Controllable Language Modelscited by 15
  11. 2022Recommendation Systems with Distribution-Free Reliability Guaranteescited by 13
  12. 2023On the Expected Size of Conformal Prediction Setscited by 13
  13. 2024Semi-Supervised Risk Control via Prediction-Powered Inferencecited by 13
  14. 2023Estimating Uncertainty in Multimodal Foundation Models using Public Internet Datacited by 10
  15. 2025Synthetic-Powered Predictive Inferencecited by 9
  16. 2021Root-finding Approaches for Computing Conformal Prediction Setcited by 8
  17. 2023Knowing When to Stop: Delay-Adaptive Spiking Neural Network Classifiers with Reliability Guaranteescited by 8
  18. 2023SymmPI: Predictive Inference for Data with Group Symmetriescited by 8
  19. 2025Direct Prediction Set Minimization via Bilevel Conformal Classifier Trainingcited by 8
  20. 2023Empirically Validating Conformal Prediction on Modern Vision Architectures Under Distribution Shift and Long-tailed Datacited by 7
  21. 2022Calibration of Natural Language Understanding Models with Venn--ABERS Predictorscited by 6
  22. 2023Conformalizing Machine Translation Evaluationcited by 5
  23. 2024Coverage-Guaranteed Prediction Sets for Out-of-Distribution Datacited by 5
  24. 2022Distribution-Free Finite-Sample Guarantees and Split Conformal Predictioncited by 4
  25. 2023Conformalization of Sparse Generalized Linear Modelscited by 4
  26. 2024Uncertainty Quantification for Neurosymbolic Programs via Compositional Conformal Predictioncited by 4
  27. 2024CHAMP: Conformalized 3D Human Multi-Hypothesis Pose Estimatorscited by 4
  28. 2025Adaptive Coverage Policies in Conformal Predictioncited by 4
  29. 2022A novel Deep Learning approach for one-step Conformal Prediction approximationcited by 3
  30. 2023Conformal Loss-Controlling Predictioncited by 3
  31. 2022ACon2: Adaptive Conformal Consensus for Provable Blockchain Oraclescited by 2
  32. 2025MetaSTNet: Multimodal Meta-learning for Cellular Traffic Conformal Predictioncited by 2
  33. 2025Another Fit Bites the Dust: Conformal Prediction as a Calibration Standard for Machine Learning in High-Energy Physicscited by 2
  34. 2022A Confidence Machine for Sparse High-Order Interaction Modelcited by 1
  35. 2025Semi-Supervised Conformal Prediction With Unlabeled Nonconformity Scorecited by 1
  36. 2025Optimizing In-Context Learning for Efficient Full Conformal Predictioncited by 1
  37. 2026CAOS: Conformal Aggregation of One-Shot Predictorscited by 1
  38. 2025Adept: Annotation-Denoising Auxiliary Tasks with Discrete Cosine Transform Map and Keypoint for Human-Centric Pretrainingcited by 0

50 most cited of 53 All 53 (JSON)