mlsanityData to 7 Oct 2026

Prediction Sets for High-Dimensional Mixture of Experts Models

arXiv 2210.16710 ↗First posted 30 Oct 2022math.ST stat.ME stat.MLcited by 8

On the map: Mixture-of-experts → MoE theory and gating functions

Abstract (arXiv)

Large datasets make it possible to build predictive models that can capture heterogenous relationships between the response variable and features. The mixture of high-dimensional linear experts model posits that observations come from a mixture of high-dimensional linear regression models, where the mixture weights are themselves feature-dependent. In this paper, we show how to construct valid prediction sets for an ℓ1-penalized mixture of experts model in the high-dimensional setting. We make use of a debiasing procedure to account for the bias induced by the penalization and propose a novel strategy for combining intervals to form a prediction set with coverage guarantees in the mixture setting. Synthetic examples and an application to the prediction of critical temperatures of superconducting materials show our method to have reliable practical performance.

Key facts. Cited by 8 papers in the map (data through 7 Oct 2026). Placed in MoE theory and gating functions, one of 12 directions in Mixture-of-experts. Builds on 20 of its 64 references that are in the map. First posted on arXiv 30 Oct 2022.

Builds on

20 of 64 references.

References · 20

  1. 2016Distribution-Free Predictive Inference For Regressioncited by 657
  2. 2019Conformalized Quantile Regressioncited by 620
  3. 2014Statistical guarantees for the EM algorithm: From population to sample-based analysiscited by 131
  4. 2013Confidence Intervals and Hypothesis Testing for High-Dimensional Regressioncited by 78
  5. 2021A Practical Survey on Faster and Lighter Transformerscited by 73
  6. 2016Approximate Residual Balancing: De-Biased Inference of Average Treatment Effects in High Dimensionscited by 50
  7. 2019With Malice Towards None: Assessing Uncertainty via Equalized Coveragecited by 43
  8. 2015De-biasing the Lasso: Optimal Sample Size for Gaussian Designscited by 35
  9. 2019EM Converges for a Mixture of Many Linear Regressionscited by 27
  10. 2018Global Convergence of EM Algorithm for Mixtures of Two Component Linear Regressioncited by 25
  11. 2015Regularized EM Algorithms: A Unified Framework and Statistical Guaranteescited by 19
  12. 2017Estimating the Coefficients of a Mixture of Two Linear Regressions by Expectation Maximizationcited by 19
8 more
  1. 2013Hypothesis Testing in High-Dimensional Regression under the Gaussian Random Design Model: Asymptotic Theorycited by 18
  2. 2016Linear Hypothesis Testing in Dense High-Dimensional Linear Modelscited by 8
  3. 2014High Dimensional Expectation-Maximization Algorithm: Statistical Optimization and Asymptotic Normalitycited by 6
  4. 2020Estimation, Confidence Intervals, and Large-Scale Hypotheses Testing for High-Dimensional Mixed Linear Regressioncited by 5
  5. 2017A Flexible Framework for Hypothesis Testing in High-dimensionscited by 4
  6. 2019Group Inference in High Dimensions with Applications to Hierarchical Testingcited by 4
  7. 2020Modeling Cell Populations Measured By Flow Cytometry With Covariates Using Sparse Mixture of Regressionscited by 2
  8. 2019Single Point Transductive Predictioncited by 1

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 2; 2024: 2; 2025: 0; 2026: 4.

8 papers.

  1. 2023Conformal Prediction for Deep Classifier via Label Rankingcited by 52
  2. 2024Length Optimization in Conformal Predictioncited by 39
  3. 2024Conformal Prediction with Learned Featurescited by 23
  4. 2023Statistical-Computational Tradeoffs in Mixed Sparse Linear Regressioncited by 7
  5. 2026Revisiting Incremental Stochastic Majorization-Minimization Algorithms with Applications to Mixture of Expertscited by 0
  6. 2026Softmax is not Enough (for Adaptive Conformal Classification)cited by 0
  7. 2026Conformal Reliability: A New Evaluation Metric for Conditional Generationcited by 0
  8. 2026Optimal Mixture-of-Experts Model Averaging for Conditional Generative Modelscited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2019Regularized Estimation and Feature Selection in Mixtures of Generalized Linear Experts
  2. 2021Non-asymptotic model selection in block-diagonal mixture of polynomial experts models
  3. 2021A non-asymptotic approach for model selection via penalization in high-dimensional mixture of experts models
  4. 2025Mixture of Many Zero-Compute Experts: A High-Rate Quantization Theory Perspective
  5. 2020A similarity-based Bayesian mixture-of-experts model
  6. 2024Generalization Error Analysis for Sparse Mixture-of-Experts: A Preliminary Study
  7. 2025Anchor-MoE: A Mean-Anchored Mixture of Experts For Probabilistic Regression
  8. 2024Learning Mixtures of Experts with EM: A Mirror Descent Perspective