# Distribution-Free Prediction Sets for Two-Layer Hierarchical Models

arXiv: https://arxiv.org/abs/1809.07441 · HTML: https://mlsanity.com/paper/1809.07441/

## Key facts

- Cited by 35 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Sep 2018.

## Built on by (most cited first, 30 of 35)

- [Training Uncertainty-Aware Classifiers with Conformalized Deep Learning](https://mlsanity.com/paper/2205.05878/) (2022)
- [Distribution Free Prediction Sets for Node Classification](https://mlsanity.com/paper/2211.14555/) (2022)
- [Conformal Prediction: A Data Perspective](https://mlsanity.com/paper/2410.06494/) (2024)
- [PAC Prediction Sets Under Label Shift](https://mlsanity.com/paper/2310.12964/) (2023)
- [PAC Prediction Sets for Meta-Learning](https://mlsanity.com/paper/2207.02440/) (2022)
- [Conformal Inference under High-Dimensional Covariate Shifts via Likelihood-Ratio Regularization](https://mlsanity.com/paper/2502.13030/) (2025)
- [Conformal online model aggregation](https://mlsanity.com/paper/2403.15527/) (2024)
- [Multi-Source Conformal Inference Under Distribution Shift](https://mlsanity.com/paper/2405.09331/) (2024)
- [Conformal Frequency Estimation using Discrete Sketched Data with Coverage for Distinct Queries](https://mlsanity.com/paper/2211.04612/) (2022)
- [Conformal Structured Prediction](https://mlsanity.com/paper/2410.06296/) (2024)
- [Synthetic-Powered Predictive Inference](https://mlsanity.com/paper/2505.13432/) (2025)
- [Empirically Validating Conformal Prediction on Modern Vision Architectures Under Distribution Shift and Long-tailed Data](https://mlsanity.com/paper/2307.01088/) (2023)
- [SymmPI: Predictive Inference for Data with Group Symmetries](https://mlsanity.com/paper/2312.16160/) (2023)
- [Distribution-Free Conformal Joint Prediction Regions for Neural Marked Temporal Point Processes](https://mlsanity.com/paper/2401.04612/) (2024)
- [Minimax Regret Learning for Data with Heterogeneous Subgroups](https://mlsanity.com/paper/2405.01709/) (2024)
- [Conditional Conformal Risk Adaptation](https://mlsanity.com/paper/2504.07611/) (2025)
- [Conformal Information Pursuit for Interactively Guiding Large Language Models](https://mlsanity.com/paper/2507.03279/) (2025)
- [Conformal Prediction for Hierarchical Data](https://mlsanity.com/paper/2411.13479/) (2024)
- [Distribution-Free Finite-Sample Guarantees and Split Conformal Prediction](https://mlsanity.com/paper/2210.14735/) (2022)
- [Predictive Inference in Multi-environment Scenarios](https://mlsanity.com/paper/2403.16336/) (2024)
- [Uncertainty quantification in metric spaces](https://mlsanity.com/paper/2405.05110/) (2024)
- [JANET: Joint Adaptive predictioN-region Estimation for Time-series](https://mlsanity.com/paper/2407.06390/) (2024)
- [Design-based conformal prediction](https://mlsanity.com/paper/2303.01422/) (2023)
- [Uncertainty Quantification With Multiple Sources](https://mlsanity.com/paper/2405.06479/) (2024)
- [Conformal and kNN Predictive Uncertainty Quantification Algorithms in Metric Spaces](https://mlsanity.com/paper/2507.15741/) (2025)
- [A Confidence Machine for Sparse High-Order Interaction Model](https://mlsanity.com/paper/2205.14317/) (2022)
- [Conformal Inference for Invariant Risk Minimization](https://mlsanity.com/paper/2305.12686/) (2023)
- [Adaptive Conformal Prediction via Bayesian Uncertainty Weighting for Hierarchical Healthcare Data](https://mlsanity.com/paper/2601.01223/) (2026)
- [Online Conformal Prediction for Non-Exchangeable Panel Data](https://mlsanity.com/paper/2605.17705/) (2026)
- [Generalized Hierarchical Conformal Prediction](https://mlsanity.com/paper/2608.15500/) (2026)

## Abstract (arXiv)

We consider the problem of constructing distribution-free prediction sets for data from two-layer hierarchical distributions. For iid data, prediction sets can be constructed using the method of conformal prediction. The validity of conformal prediction hinges on the exchangeability of the data, which does not hold when groups of observations come from distinct distributions, such as multiple observations on each patient in a medical database. We extend conformal methods to this hierarchical setting. We develop CDF pooling, single subsampling, and repeated subsampling approaches to construct prediction sets in unsupervised and supervised settings. We compare these approaches in terms of coverage and average set size. If asymptotic coverage is acceptable, we recommend CDF pooling for its balance between empirical coverage and average set size. If we desire coverage guarantees, then we recommend the repeated subsampling approach.