mlsanityData to 14 Oct 2026

EM Converges for a Mixture of Many Linear Regressions

arXiv 1905.12106 ↗First posted 28 May 2019cs.LG stat.MLcited by 27

Not on the map. Why?

Abstract (arXiv)

We study the convergence of the Expectation-Maximization (EM) algorithm for mixtures of linear regressions with an arbitrary number k of components. We show that as long as signal-to-noise ratio (SNR) is tildeOmega(k), well-initialized EM converges to the true regression parameters. Previous results for k ≥ 3 have only established local convergence for the noiseless setting, i.e., where SNR is infinitely large. Our results enlarge the scope to the environment with noises, and notably, we establish a statistical error rate that is independent of the norm (or pairwise distance) of the regression parameters. In particular, our results imply exact recovery as σ rightarrow 0, in contrast to most previous local convergence results for EM, where the statistical error scaled with the norm of parameters. Standard moment-method approaches may be applied to guarantee we are in the region where our local convergence guarantees apply.

Key facts. Cited by 27 papers in the map (data through 14 Oct 2026). First posted on arXiv 28 May 2019.

Builds on

No reference list yet.

Built on by

Cited by year (2026 to 14 Oct)

Citing papers per year: 2021: 4; 2022: 4; 2023: 10; 2024: 5; 2025: 3; 2026: 1.

27 papers.

Sort:

By area:

  1. 2021RL for Latent MDPs: Regret Guarantees and a Lower Boundcited by 70
  2. 2023Demystifying Softmax Gating Function in Gaussian Mixture of Expertscited by 33
  3. 2023Statistical Perspective of Top-K Sparse Softmax Gating Mixture of Expertscited by 29
  4. 2021Statistical Query Lower Bounds for List-Decodable Linear Regressioncited by 25
  5. 2023A General Theory for Softmax Gating Multinomial Logistic Mixture of Expertscited by 23
  6. 2023Towards Convergence Rates for Parameter Estimation in Gaussian-gated Mixture of Expertscited by 17
  7. 2022Prediction Sets for High-Dimensional Mixture of Experts Modelscited by 8
  8. 2021A Wasserstein Minimax Framework for Mixed Linear Regressioncited by 7
  9. 2022What Makes A Good Fisherman? Linear Regression under Self-Selection Biascited by 7
  10. 2023Towards the Theory of Unsupervised Federated Learning: Non-asymptotic Analysis of Federated EM Algorithmscited by 7
  11. 2024Multi-Model 3D Registration: Finding Multiple Moving Objects in Cluttered Point Cloudscited by 7
  12. 2024Implicit High-Order Moment Tensor Estimation and Learning Latent Variable Modelscited by 6
15 more
  1. 2021Uniform Consistency in Nonparametric Mixture Modelscited by 5
  2. 2023Imbalanced Mixed Linear Regressioncited by 3
  3. 2022EM's Convergence in Gaussian Latent Tree Modelscited by 2
  4. 2023Sharp analysis of EM for learning mixtures of pairwise differencescited by 1
  5. 2023EM for Mixture of Linear Regression with Clustered Datacited by 1
  6. 2023Global Convergence of Online Identification for Mixed Linear Regressioncited by 1
  7. 2024In-context Learning for Mixture of Linear Regressions: Existence, Generalization and Training Dynamicscited by 1
  8. 2025On Minimax Estimation of Parameters in Softmax-Contaminated Mixture of Expertscited by 1
  9. 2022Embedding Functional Data: Multidimensional Scaling and Manifold Learningcited by 0
  10. 2023On the robust learning mixtures of linear regressionscited by 0
  11. 2024Sample-Efficient Linear Regression with Self-Selection Biascited by 0
  12. 2024Fast convergence of a Federated Expectation-Maximization Algorithmcited by 0
  13. 2025Learning a Class of Mixed Linear Regressions: Global Convergence under General Data Conditionscited by 0
  14. 2025Linearly Solving Robust Rotation Estimationcited by 0
  15. 2026Rethinking Multinomial Logistic Mixture of Experts with Sigmoid Gating Functioncited by 0