# A Unified View on Learning Unnormalized Distributions via Noise-Contrastive Estimation

arXiv: https://arxiv.org/abs/2409.18209 · HTML: https://mlsanity.com/paper/2409.18209/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 36 references that are in the map.
- First posted on arXiv 26 Sep 2024.

## Builds on (most cited first, 18 of 18)

- [Sliced Score Matching: A Scalable Approach to Density and Score Estimation](https://mlsanity.com/paper/1905.07088/) (2019)
- [How to Train Your Energy-Based Models](https://mlsanity.com/paper/2101.03288/) (2021)
- [Interaction Screening: Efficient and Sample-Optimal Learning of Ising Models](https://mlsanity.com/paper/1605.07252/) (2016)
- [Conditional Noise-Contrastive Estimation of Unnormalised Models](https://mlsanity.com/paper/1806.03664/) (2018)
- [Efficient Learning of Discrete Graphical Models](https://mlsanity.com/paper/1902.00600/) (2019)
- [Estimating Density Models with Truncation Boundaries using Score Matching](https://mlsanity.com/paper/1910.03834/) (2019)
- [Provable benefits of score matching](https://mlsanity.com/paper/2306.01993/) (2023)
- [Analyzing and Improving the Optimization Landscape of Noise-Contrastive Estimation](https://mlsanity.com/paper/2110.11271/) (2021)
- [Provable benefits of annealing for estimating normalizing constants: Importance Sampling, Noise-Contrastive Estimation, and beyond](https://mlsanity.com/paper/2310.03902/) (2023)
- [The Optimal Noise in Noise-Contrastive Learning Is Not What You Think](https://mlsanity.com/paper/2203.01110/) (2022)
- [A Computationally Efficient Method for Learning Exponential Family Distributions](https://mlsanity.com/paper/2110.15397/) (2021)
- [On Learning Continuous Pairwise Markov Random Fields](https://mlsanity.com/paper/2010.15031/) (2020)
- [Analysis of Noise Contrastive Estimation from the Perspective of Asymptotic Variance](https://mlsanity.com/paper/1808.07983/) (2018)
- [Learning Continuous Exponential Families Beyond Gaussian](https://mlsanity.com/paper/2102.09198/) (2021)
- [Pitfalls of Gaussians as a noise distribution in NCE](https://mlsanity.com/paper/2210.00189/) (2022)
- [Learning Unnormalized Statistical Models via Compositional Optimization](https://mlsanity.com/paper/2306.07485/) (2023)
- [On Computationally Efficient Learning of Exponential Family Distributions](https://mlsanity.com/paper/2309.06413/) (2023)
- [The Poisson transform for unnormalised statistical models](https://mlsanity.com/paper/1406.2839/) (2014)

## Built on by (most cited first, 2 of 2)

- [Contrastive Predictive Coding Done Right for Mutual Information Estimation](https://mlsanity.com/paper/2510.25983/) (2025)
- [Projective Maximum Entropy: Universality and Acceptance-Region Calibration](https://mlsanity.com/paper/2607.16547/) (2026)

## Abstract (arXiv)

This paper studies a family of estimators based on noise-contrastive estimation (NCE) for learning unnormalized distributions. The main contribution of this work is to provide a unified perspective on various methods for learning unnormalized distributions, which have been independently proposed and studied in separate research communities, through the lens of NCE. This unified view offers new insights into existing estimators. Specifically, for exponential families, we establish the finite-sample convergence rates of the proposed estimators under a set of regularity assumptions, most of which are new.