# Optimal convex M-estimation via score matching

arXiv: https://arxiv.org/abs/2403.16688 · HTML: https://mlsanity.com/paper/2403.16688/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 25 Mar 2024.

## Built on by (most cited first, 9 of 9)

- [A Sharp Convergence Theory for The Probability Flow ODEs of Diffusion Models](https://mlsanity.com/paper/2408.02320/) (2024)
- [Low-dimensional adaptation of diffusion models: Convergence in total variation](https://mlsanity.com/paper/2501.12982/) (2025)
- [Stochastic Runge-Kutta Methods: Provable Acceleration of Diffusion Models](https://mlsanity.com/paper/2410.04760/) (2024)
- [Dimension-Free Convergence of Diffusion Models for Approximate Gaussian Mixtures](https://mlsanity.com/paper/2504.05300/) (2025)
- [Nonparametric inference under shape constraints: past, present and future](https://mlsanity.com/paper/2509.26040/) (2025)
- [Learning density ratios in causal inference using Bregman-Riesz regression](https://mlsanity.com/paper/2510.16127/) (2025)
- [Distributional Shrinkage I: Universal Denoiser Beyond Tweedie's Formula](https://mlsanity.com/paper/2511.09500/) (2025)
- [Distributional Shrinkage II: Higher-Order Scores Encode Brenier Map](https://mlsanity.com/paper/2512.09295/) (2025)
- [Instance-Optimal Adaptive Location Estimation via Multiscale Mid-Summaries](https://mlsanity.com/paper/2609.20749/) (2026)

## Abstract (arXiv)

In the context of linear regression, we construct a data-driven convex loss function with respect to which empirical risk minimisation yields optimal asymptotic variance in the downstream estimation of the regression coefficients. At the population level, the negative derivative of the optimal convex loss is the best decreasing approximation of the derivative of the log-density of the noise distribution. This motivates a fitting process via a nonparametric extension of score matching, corresponding to a log-concave projection of the noise distribution with respect to the Fisher divergence. At the sample level, our semiparametric estimator is computationally efficient, and we prove that it attains the minimal asymptotic covariance among all convex M-estimators. As an example of a non-log-concave setting, the optimal convex loss function for Cauchy errors is Huber-like, and our procedure yields asymptotic efficiency greater than 0.87 relative to the maximum likelihood estimator of the regression coefficients that uses oracle knowledge of this error distribution. In this sense, we provide robustness and facilitate computation without sacrificing much statistical efficiency. Numerical experiments using our accompanying R package 'asm' confirm the practical merits of our proposal.