# A Computationally Efficient Method for Learning Exponential Family Distributions

arXiv: https://arxiv.org/abs/2110.15397 · HTML: https://mlsanity.com/paper/2110.15397/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 70 references that are in the map.
- First posted on arXiv 28 Oct 2021.

## Builds on (most cited first, 25 of 25)

- [A Kernelized Stein Discrepancy for Goodness-of-fit Tests and Model Evaluation](https://mlsanity.com/paper/1602.03253/) (2016)
- [A Kernel Test of Goodness of Fit](https://mlsanity.com/paper/1602.02964/) (2016)
- [Telescoping Density-Ratio Estimation](https://mlsanity.com/paper/2006.12204/) (2020)
- [Minimum Stein Discrepancy Estimators](https://mlsanity.com/paper/1906.08283/) (2019)
- [Interaction Screening: Efficient and Sample-Optimal Learning of Ising Models](https://mlsanity.com/paper/1605.07252/) (2016)
- [Learning Graphical Models Using Multiplicative Weights](https://mlsanity.com/paper/1706.06274/) (2017)
- [Density Estimation in Infinite Dimensional Exponential Families](https://mlsanity.com/paper/1312.3516/) (2013)
- [Learning deep kernels for exponential family densities](https://mlsanity.com/paper/1811.08357/) (2018)
- [Exponential Family Estimation via Adversarial Dynamics Embedding](https://mlsanity.com/paper/1904.12083/) (2019)
- [Efficient Statistics, in High Dimensions, from Truncated Samples](https://mlsanity.com/paper/1809.03986/) (2018)
- [On Graphical Models via Univariate Exponential Family Distributions](https://mlsanity.com/paper/1301.4183/) (2013)
- [Efficient Learning of Discrete Graphical Models](https://mlsanity.com/paper/1902.00600/) (2019)
- [Estimation of High-Dimensional Graphical Models Using Regularized Score Matching](https://mlsanity.com/paper/1507.00433/) (2015)
- [Learning Some Popular Gaussian Graphical Models without Condition Number Bounds](https://mlsanity.com/paper/1905.01282/) (2019)
- [Gradient-free Hamiltonian Monte Carlo with Efficient Kernel Exponential Families](https://mlsanity.com/paper/1506.02564/) (2015)
- [Outlier-Robust Learning of Ising Models Under Dobrushin's Condition](https://mlsanity.com/paper/2102.02171/) (2021)
- [Efficient and principled score estimation with Nystr"om kernel exponential families](https://mlsanity.com/paper/1705.08360/) (2017)
- [Fisher Efficient Inference of Intractable Models](https://mlsanity.com/paper/1805.07454/) (2018)
- [On Learning Continuous Pairwise Markov Random Fields](https://mlsanity.com/paper/2010.15031/) (2020)
- [Learning Latent Variable Gaussian Graphical Models](https://mlsanity.com/paper/1406.2721/) (2014)
- [Estimating Differential Latent Variable Graphical Models with Applications to Brain Connectivity](https://mlsanity.com/paper/1909.05892/) (2019)
- [On Semiparametric Exponential Family Graphical Models](https://mlsanity.com/paper/1412.8697/) (2014)
- [Learning Continuous Exponential Families Beyond Gaussian](https://mlsanity.com/paper/2102.09198/) (2021)
- [Vector-Space Markov Random Fields via Exponential Families](https://mlsanity.com/paper/1505.05117/) (2015)
- [A Likelihood Ratio Framework for High Dimensional Semiparametric Regression](https://mlsanity.com/paper/1412.2295/) (2014)

## Built on by (most cited first, 10 of 10)

- [Provable benefits of score matching](https://mlsanity.com/paper/2306.01993/) (2023)
- [Bilinear Exponential Family of MDPs: Frequentist Regret Bound with Tractable Exploration and Planning](https://mlsanity.com/paper/2210.02087/) (2022)
- [On counterfactual inference with unobserved confounding](https://mlsanity.com/paper/2211.08209/) (2022)
- [Efficient Statistics With Unknown Truncation, Polynomial Time Algorithms, Beyond Gaussians](https://mlsanity.com/paper/2410.01656/) (2024)
- [On Computationally Efficient Learning of Exponential Family Distributions](https://mlsanity.com/paper/2309.06413/) (2023)
- [A Unified View on Learning Unnormalized Distributions via Noise-Contrastive Estimation](https://mlsanity.com/paper/2409.18209/) (2024)
- [Private Statistical Estimation via Truncation](https://mlsanity.com/paper/2505.12541/) (2025)
- [DRIVE: Dynamic Rule Inference and Verified Evaluation for Constraint-Aware Autonomous Driving](https://mlsanity.com/paper/2508.04066/) (2025)
- [Finite Sample Bounds for Learning with Score Matching](https://mlsanity.com/paper/2605.14168/) (2026)
- [Causal Inference for Sequential Settings under Interference and Latent Confounding](https://mlsanity.com/paper/2607.14940/) (2026)

## Abstract (arXiv)

We consider the question of learning the natural parameters of a k parameter minimal exponential family from i.i.d. samples in a computationally and statistically efficient manner. We focus on the setting where the support as well as the natural parameters are appropriately bounded. While the traditional maximum likelihood estimator for this class of exponential family is consistent, asymptotically normal, and asymptotically efficient, evaluating it is computationally hard. In this work, we propose a computationally efficient estimator that is consistent as well as asymptotically normal under mild conditions. We provide finite sample guarantees to achieve an (ℓ2) error of α in the parameter estimation with sample complexity O(poly(k/α)) and computational complexity O(poly(k/α)). To establish these results, we show that, at the population level, our method can be viewed as the maximum likelihood estimation of a re-parameterized distribution belonging to the same class of exponential family.