# Mixture-of-experts Wishart model for covariance matrices with an application to Cancer drug screening

arXiv: https://arxiv.org/abs/2602.13888 · HTML: https://mlsanity.com/paper/2602.13888/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in MoE theory and gating functions, one of 12 directions in Mixture-of-experts.
- Builds on 8 of its 36 references that are in the map.
- First posted on arXiv 14 Feb 2026.

## Placed in

- Mixture-of-experts / [MoE theory and gating functions](https://mlsanity.com/moe/moe-theory-and-gating-functions/)

## Builds on (most cited first, 8 of 8)

- [Practical Bayesian model evaluation using leave-one-out cross-validation and WAIC](https://mlsanity.com/paper/1507.04544/) (2015)
- [Spectral Mixture Kernels for Multi-Output Gaussian Processes](https://mlsanity.com/paper/1709.01298/) (2017)
- [High-dimensional sparse classification using exponential weighting with empirical hinge loss](https://mlsanity.com/paper/2312.12952/) (2023)
- [Concentration of a sparse Bayesian model with Horseshoe prior in estimating high-dimensional precision matrix](https://mlsanity.com/paper/2406.14269/) (2024)
- [Concentration properties of fractional posterior in 1-bit matrix completion](https://mlsanity.com/paper/2404.08969/) (2024)
- [High-dimensional Bayesian Tobit regression for censored response with Horseshoe prior](https://mlsanity.com/paper/2505.08288/) (2025)
- [Model-based clustering for covariance matrices via penalized Wishart mixture models](https://mlsanity.com/paper/2408.17040/) (2024)
- [Handling bounded response in high dimensions: a Horseshoe prior Bayesian Beta regression approach](https://mlsanity.com/paper/2505.22211/) (2025)

## Abstract (arXiv)

Covariance matrices arise naturally in different scientific fields, including finance, genomics, and neuroscience, where they encode dependence structures and reveal essential features of complex multivariate systems. In this work, we introduce a comprehensive Bayesian framework for analyzing heterogeneous covariance data through both classical mixture models and a novel mixture-of-experts Wishart (MoE-Wishart) model. The proposed MoE-Wishart model extends standard Wishart mixtures by allowing mixture weights to depend on predictors through a multinomial logistic gating network. This formulation enables the model to capture complex, nonlinear heterogeneity in covariance structures and to adapt subpopulation membership probabilities to covariate-dependent patterns. To perform inference, we develop an efficient Gibbs-within-Metropolis-Hastings sampling algorithm tailored to the geometry of the Wishart likelihood and the gating network. We additionally derive an Expectation-Maximization algorithm for maximum likelihood estimation in the mixture-of-experts setting. Extensive simulation studies demonstrate that the proposed Bayesian and maximum likelihood estimators achieve accurate subpopulation recovery and estimation under a range of heterogeneous covariance scenarios. Finally, we present an innovative application of our methodology to a challenging dataset: cancer drug sensitivity profiles, illustrating the ability of the MoE-Wishart model to leverage covariance across drug dosages and replicate measurements. Our methods are implemented in the textttR package textttmoewishart available at https://github.com/zhizuio/moewishart .