# Efficient Algorithms for Estimating the Parameters of Mixed Linear Regression Models

arXiv: https://arxiv.org/abs/2105.05953 · HTML: https://mlsanity.com/paper/2105.05953/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 34 references that are in the map.
- First posted on arXiv 12 May 2021.

## Builds on (most cited first, 13 of 13)

- [The Landscape of Empirical Risk for Non-convex Losses](https://mlsanity.com/paper/1607.06534/) (2016)
- [Convergence Analysis of Alternating Direction Method of Multipliers for a Family of Nonconvex Problems](https://mlsanity.com/paper/1410.1390/) (2014)
- [Global Convergence of EM Algorithm for Mixtures of Two Component Linear Regression](https://mlsanity.com/paper/1810.05752/) (2018)
- [Spectral Experts for Estimating Mixtures of Linear Regressions](https://mlsanity.com/paper/1306.3729/) (2013)
- [Fisher Vectors Derived from Hybrid Gaussian-Laplacian Mixture Models for Image Annotation](https://mlsanity.com/paper/1411.7399/) (2014)
- [Iterative Least Trimmed Squares for Mixed Linear Regression](https://mlsanity.com/paper/1902.03653/) (2019)
- [Solving Non-Convex Non-Differentiable Min-Max Games using Proximal Gradient Method](https://mlsanity.com/paper/2003.08093/) (2020)
- [Solving a class of non-convex min-max games using adaptive momentum methods](https://mlsanity.com/paper/2104.12676/) (2021)
- [Global Convergence of Least Squares EM for Demixing Two Log-Concave Densities](https://mlsanity.com/paper/1906.06776/) (2019)
- [On the Behavior of the Expectation-Maximization Algorithm for Mixture Models](https://mlsanity.com/paper/1809.08705/) (2018)
- [Convergence of Parameter Estimates for Regularized Mixed Linear Regression Models](https://mlsanity.com/paper/1903.09235/) (2019)
- [Training generative networks using random discriminators](https://mlsanity.com/paper/1904.09775/) (2019)
- [A Block Successive Upper Bound Minimization Method of Multipliers for Linearly Constrained Convex Optimization](https://mlsanity.com/paper/1401.7079/) (2014)

## Built on by (most cited first, 1 of 1)

- [Imbalanced Mixed Linear Regression](https://mlsanity.com/paper/2301.12559/) (2023)

## Abstract (arXiv)

Mixed linear regression (MLR) model is among the most exemplary statistical tools for modeling non-linear distributions using a mixture of linear models. When the additive noise in MLR model is Gaussian, Expectation-Maximization (EM) algorithm is a widely-used algorithm for maximum likelihood estimation of MLR parameters. However, when noise is non-Gaussian, the steps of EM algorithm may not have closed-form update rules, which makes EM algorithm impractical. In this work, we study the maximum likelihood estimation of the parameters of MLR model when the additive noise has non-Gaussian distribution. In particular, we consider the case that noise has Laplacian distribution and we first show that unlike the the Gaussian case, the resulting sub-problems of EM algorithm in this case does not have closed-form update rule, thus preventing us from using EM in this case. To overcome this issue, we propose a new algorithm based on combining the alternating direction method of multipliers (ADMM) with EM algorithm idea. Our numerical experiments show that our method outperforms the EM algorithm in statistical accuracy and computational time in non-Gaussian noise case.