# Imbalanced Mixed Linear Regression

arXiv: https://arxiv.org/abs/2301.12559 · HTML: https://mlsanity.com/paper/2301.12559/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 24 of its 57 references that are in the map.
- First posted on arXiv 29 Jan 2023.

## Builds on (most cited first, 24 of 24)

- [Statistical guarantees for the EM algorithm: From population to sample-based analysis](https://mlsanity.com/paper/1408.2156/) (2014)
- [Learning from Untrusted Data](https://mlsanity.com/paper/1611.02315/) (2016)
- [List-Decodable Linear Regression](https://mlsanity.com/paper/1905.05679/) (2019)
- [Ten Steps of EM Suffice for Mixtures of Two Gaussians](https://mlsanity.com/paper/1609.00368/) (2016)
- [Learning Mixtures of Linear Regressions with Nearly Optimal Complexity](https://mlsanity.com/paper/1802.07895/) (2018)
- [EM Converges for a Mixture of Many Linear Regressions](https://mlsanity.com/paper/1905.12106/) (2019)
- [Solving a Mixture of Many Random Linear Equations by Tensor Decomposition and Alternating Minimization](https://mlsanity.com/paper/1608.05749/) (2016)
- [Learning Mixtures of Linear Regressions in Subexponential Time via Fourier Moments](https://mlsanity.com/paper/1912.07629/) (2019)
- [Global Convergence of EM Algorithm for Mixtures of Two Component Linear Regression](https://mlsanity.com/paper/1810.05752/) (2018)
- [On the Minimax Optimality of the EM Algorithm for Learning Two-Component Mixed Linear Regression](https://mlsanity.com/paper/2006.02601/) (2020)
- [Spectral Experts for Estimating Mixtures of Linear Regressions](https://mlsanity.com/paper/1306.3729/) (2013)
- [Estimating the Coefficients of a Mixture of Two Linear Regressions by Expectation Maximization](https://mlsanity.com/paper/1704.08231/) (2017)
- [Globally-convergent Iteratively Reweighted Least Squares for Robust Regression Problems](https://mlsanity.com/paper/2006.14211/) (2020)
- [Iterative Least Trimmed Squares for Mixed Linear Regression](https://mlsanity.com/paper/1902.03653/) (2019)
- [Alternating Minimization Converges Super-Linearly for Mixed Linear Regression](https://mlsanity.com/paper/2004.10914/) (2020)
- [On Learning Mixture of Linear Regressions in the Non-Realizable Setting](https://mlsanity.com/paper/2205.13166/) (2022)
- [Global Linear and Local Superlinear Convergence of IRLS for Non-Smooth Robust Regression](https://mlsanity.com/paper/2208.11846/) (2022)
- [A Wasserstein Minimax Framework for Mixed Linear Regression](https://mlsanity.com/paper/2106.07537/) (2021)
- [Hyperplane Clustering Via Dual Principal Component Pursuit](https://mlsanity.com/paper/1706.01604/) (2017)
- [Estimation, Confidence Intervals, and Large-Scale Hypotheses Testing for High-Dimensional Mixed Linear Regression](https://mlsanity.com/paper/2011.03598/) (2020)
- [A Nonparametric Maximum Likelihood Approach to Mixture of Regression](https://mlsanity.com/paper/2108.09816/) (2021)
- [Supervised clustering of high dimensional data using regularized mixture modeling](https://mlsanity.com/paper/2007.09720/) (2020)
- [Robust structured heterogeneity analysis approach for high-dimensional data](https://mlsanity.com/paper/2211.16475/) (2022)
- [Efficient Algorithms for Estimating the Parameters of Mixed Linear Regression Models](https://mlsanity.com/paper/2105.05953/) (2021)

## Built on by (most cited first, 3 of 3)

- [Mixed Regression via Approximate Message Passing](https://mlsanity.com/paper/2304.02229/) (2023)
- [Global Convergence of Online Identification for Mixed Linear Regression](https://mlsanity.com/paper/2311.18506/) (2023)
- [Learning a Class of Mixed Linear Regressions: Global Convergence under General Data Conditions](https://mlsanity.com/paper/2503.18500/) (2025)

## Abstract (arXiv)

We consider the problem of mixed linear regression (MLR), where each observed sample belongs to one of K unknown linear models. In practical applications, the proportions of the K components are often imbalanced. Unfortunately, most MLR methods do not perform well in such settings. Motivated by this practical challenge, in this work we propose Mix-IRLS, a novel, simple and fast algorithm for MLR with excellent performance on both balanced and imbalanced mixtures. In contrast to popular approaches that recover the K models simultaneously, Mix-IRLS does it sequentially using tools from robust regression. Empirically, Mix-IRLS succeeds in a broad range of settings where other methods fail. These include imbalanced mixtures, small sample sizes, presence of outliers, and an unknown number of models K. In addition, Mix-IRLS outperforms competing methods on several real-world datasets, in some cases by a large margin. We complement our empirical results by deriving a recovery guarantee for Mix-IRLS, which highlights its advantage on imbalanced mixtures.