# Connecting Score Matching, Maximum Likelihood, and Expectation-Maximization in Mixed Linear Regression

arXiv: https://arxiv.org/abs/2609.05688 · HTML: https://mlsanity.com/paper/2609.05688/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 9 of its 62 references that are in the map.
- First posted on arXiv 4 Sep 2026.

## Builds on (most cited first, 9 of 9)

- [Conditional Image Generation with Score-Based Diffusion Models](https://mlsanity.com/paper/2111.13606/) (2021)
- [Unveil Conditional Diffusion Models with Classifier-free Guidance: A Sharp Statistical Theory](https://mlsanity.com/paper/2403.11968/) (2024)
- [Blindness of score-based methods to isolated components and mixing proportions](https://mlsanity.com/paper/2008.10087/) (2020)
- [DDPM Score Matching and Distribution Learning](https://mlsanity.com/paper/2504.05161/) (2025)
- [A Score-Based Density Formula, with Applications in Diffusion Generative Models](https://mlsanity.com/paper/2408.16765/) (2024)
- [Structural Properties, Cycloid Trajectories and Non-Asymptotic Guarantees of EM Algorithm for Mixed Linear Regression](https://mlsanity.com/paper/2511.04937/) (2025)
- [Diffusion models recover accurate mixture weights despite score function insensitivity](https://mlsanity.com/paper/2607.15485/) (2026)
- [Computing Kurdyka-Lojasiewicz exponents via composition and symmetry](https://mlsanity.com/paper/2602.22553/) (2026)
- [Diffusion-based Denoising Beats Vanilla Score Matching in Parameter Estimation: A Theoretical Explanation](https://mlsanity.com/paper/2605.22950/) (2026)

## Abstract (arXiv)

We study variance-preserving diffusion of the response in mixed linear regression (MLR) with unknown mixing weights. Our analysis separates the statistical guarantees of score matching from the loss geometry and optimization signal at a fixed diffusion noise level. The KL divergence links the denoising score matching objective integrated over the diffusion path with the likelihood and a terminal discrepancy. Under mild regularity conditions and terminal schedule, the resulting estimator converges up to the ground truth parameters of MLR, and its scaled error converges to the Gaussian limit of the maximum-likelihood estimator. At a fixed scale of the diffusion noise level, we derive a decomposition linking the score matching loss to cross-entropy and Expectation-Maximization (EM) operators. This decomposition yields an EM-related low-noise gradient expansion with additional correction terms of latent variance. In the high-noise limit, we further characterize gradient descent on this limiting loss under isotropic covariance. Along fixed high signal-to-noise ratio rays, the score matching imbalance gradient and the latent-variance term tend to zero pointwise. Numerical experiments illustrate our theoretical findings and statistical guarantees.