mlsanityData to 14 Oct 2026

Big Learning Expectation Maximization

arXiv 2312.11926 ↗First posted 19 Dec 2023cs.LG stat.ME stat.MLcited by 2

Not on the map. Why?

Abstract (arXiv)

Mixture models serve as one fundamental tool with versatile applications. However, their training techniques, like the popular Expectation Maximization (EM) algorithm, are notoriously sensitive to parameter initialization and often suffer from bad local optima that could be arbitrarily worse than the optimal. To address the long-lasting bad-local-optima challenge, we draw inspiration from the recent ground-breaking foundation models and propose to leverage their underlying big learning principle to upgrade the EM. Specifically, we present the Big Learning EM (BigLearn-EM), an EM upgrade that simultaneously performs joint, marginal, and orthogonally transformed marginal matchings between data and model distributions. Through simulated experiments, we empirically show that the BigLearn-EM is capable of delivering the optimal with high probability; comparisons on benchmark clustering datasets further demonstrate its effectiveness and advantages over existing techniques. The code is available at https://github.com/YulaiCong/Big-Learning-Expectation-Maximization.

Key facts. Cited by 2 papers in the map (data through 14 Oct 2026). Builds on 25 of its 66 references that are in the map. First posted on arXiv 19 Dec 2023.

Builds on

25 of 66 references.

References · 25

  1. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  2. 2020Language Models are Few-Shot Learnerscited by 32,419
  3. 2022Training language models to follow instructions with human feedbackcited by 15,971
  4. 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,403
  5. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,585
  6. 2015Unsupervised Deep Embedding for Clustering Analysiscited by 767
  7. 2016Variational Deep Embedding: An Unsupervised and Generative Approach to Clusteringcited by 220
  8. 2019Generalized Sliced Wasserstein Distancescited by 208
  9. 2016Joint Unsupervised Learning of Deep Representations and Image Clusterscited by 205
  10. 2023One Transformer Fits All Distributions in Multi-Modal Diffusion at Scalecited by 187
  11. 2014Statistical guarantees for the EM algorithm: From population to sample-based analysiscited by 133
  12. 2019Max-Sliced Wasserstein Distance and its use for GANscited by 119
13 more
  1. 2017PacGAN: The power of two samples in generative adversarial networkscited by 104
  2. 2017Sliced Wasserstein Distance for Learning Gaussian Mixture Modelscited by 59
  3. 2016Ten Steps of EM Suffice for Mixtures of Two Gaussianscited by 40
  4. 2016Local Maxima in the Likelihood of Gaussian Mixture Models: Structural Results and Algorithmic Consequencescited by 34
  5. 2016Global analysis of Expectation Maximization for mixtures of two Gaussianscited by 31
  6. 2015Learning Mixtures of Gaussians in High Dimensionscited by 29
  7. 2023Learning Gaussian Mixtures Using the Wasserstein-Fisher-Rao Gradient Flowcited by 27
  8. 2022Continuous Mixtures of Tractable Probabilistic Modelscited by 19
  9. 2020Robust Learning of Mixtures of Gaussianscited by 15
  10. 2019Reweighted Expectation Maximizationcited by 9
  11. 2019Global Convergence of Least Squares EM for Demixing Two Log-Concave Densitiescited by 5
  12. 2021A sampling-based approach for efficient clustering in large datasetscited by 4
  13. 2013MCMC for Normalized Random Measure Mixture Modelscited by 3

Built on by

Cited by year

Citing papers per year: 2024: 1; 2025: 1.

2 papers.

  1. 2025Filling the Missings: Spatiotemporal Data Imputation by Conditional Diffusioncited by 1
  2. 2024Big Cooperative Learningcited by 0