# Prediction Sets for High-Dimensional Mixture of Experts Models

arXiv: https://arxiv.org/abs/2210.16710 · HTML: https://mlsanity.com/paper/2210.16710/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- Placed in MoE theory and gating functions, one of 12 directions in Mixture-of-experts.
- Builds on 20 of its 64 references that are in the map.
- First posted on arXiv 30 Oct 2022.

## Placed in

- Mixture-of-experts / [MoE theory and gating functions](https://mlsanity.com/moe/moe-theory-and-gating-functions/)

## Builds on (most cited first, 20 of 20)

- [Distribution-Free Predictive Inference For Regression](https://mlsanity.com/paper/1604.04173/) (2016)
- [Conformalized Quantile Regression](https://mlsanity.com/paper/1905.03222/) (2019)
- [Statistical guarantees for the EM algorithm: From population to sample-based analysis](https://mlsanity.com/paper/1408.2156/) (2014)
- [Confidence Intervals and Hypothesis Testing for High-Dimensional Regression](https://mlsanity.com/paper/1306.3171/) (2013)
- [A Practical Survey on Faster and Lighter Transformers](https://mlsanity.com/paper/2103.14636/) (2021)
- [Approximate Residual Balancing: De-Biased Inference of Average Treatment Effects in High Dimensions](https://mlsanity.com/paper/1604.07125/) (2016)
- [With Malice Towards None: Assessing Uncertainty via Equalized Coverage](https://mlsanity.com/paper/1908.05428/) (2019)
- [De-biasing the Lasso: Optimal Sample Size for Gaussian Designs](https://mlsanity.com/paper/1508.02757/) (2015)
- [EM Converges for a Mixture of Many Linear Regressions](https://mlsanity.com/paper/1905.12106/) (2019)
- [Global Convergence of EM Algorithm for Mixtures of Two Component Linear Regression](https://mlsanity.com/paper/1810.05752/) (2018)
- [Regularized EM Algorithms: A Unified Framework and Statistical Guarantees](https://mlsanity.com/paper/1511.08551/) (2015)
- [Estimating the Coefficients of a Mixture of Two Linear Regressions by Expectation Maximization](https://mlsanity.com/paper/1704.08231/) (2017)
- [Hypothesis Testing in High-Dimensional Regression under the Gaussian Random Design Model: Asymptotic Theory](https://mlsanity.com/paper/1301.4240/) (2013)
- [Linear Hypothesis Testing in Dense High-Dimensional Linear Models](https://mlsanity.com/paper/1610.02987/) (2016)
- [High Dimensional Expectation-Maximization Algorithm: Statistical Optimization and Asymptotic Normality](https://mlsanity.com/paper/1412.8729/) (2014)
- [Estimation, Confidence Intervals, and Large-Scale Hypotheses Testing for High-Dimensional Mixed Linear Regression](https://mlsanity.com/paper/2011.03598/) (2020)
- [A Flexible Framework for Hypothesis Testing in High-dimensions](https://mlsanity.com/paper/1704.07971/) (2017)
- [Group Inference in High Dimensions with Applications to Hierarchical Testing](https://mlsanity.com/paper/1909.01503/) (2019)
- [Modeling Cell Populations Measured By Flow Cytometry With Covariates Using Sparse Mixture of Regressions](https://mlsanity.com/paper/2008.11251/) (2020)
- [Single Point Transductive Prediction](https://mlsanity.com/paper/1908.02341/) (2019)

## Built on by (most cited first, 8 of 8)

- [Conformal Prediction for Deep Classifier via Label Ranking](https://mlsanity.com/paper/2310.06430/) (2023)
- [Length Optimization in Conformal Prediction](https://mlsanity.com/paper/2406.18814/) (2024)
- [Conformal Prediction with Learned Features](https://mlsanity.com/paper/2404.17487/) (2024)
- [Statistical-Computational Tradeoffs in Mixed Sparse Linear Regression](https://mlsanity.com/paper/2303.02118/) (2023)
- [Revisiting Incremental Stochastic Majorization-Minimization Algorithms with Applications to Mixture of Experts](https://mlsanity.com/paper/2601.19811/) (2026)
- [Softmax is not Enough (for Adaptive Conformal Classification)](https://mlsanity.com/paper/2602.19498/) (2026)
- [Conformal Reliability: A New Evaluation Metric for Conditional Generation](https://mlsanity.com/paper/2605.30807/) (2026)
- [Optimal Mixture-of-Experts Model Averaging for Conditional Generative Models](https://mlsanity.com/paper/2607.04360/) (2026)

## Abstract (arXiv)

Large datasets make it possible to build predictive models that can capture heterogenous relationships between the response variable and features. The mixture of high-dimensional linear experts model posits that observations come from a mixture of high-dimensional linear regression models, where the mixture weights are themselves feature-dependent. In this paper, we show how to construct valid prediction sets for an ℓ1-penalized mixture of experts model in the high-dimensional setting. We make use of a debiasing procedure to account for the bias induced by the penalization and propose a novel strategy for combining intervals to form a prediction set with coverage guarantees in the mixture setting. Synthetic examples and an application to the prediction of critical temperatures of superconducting materials show our method to have reliable practical performance.