# Bi-level Score Matching for Learning Energy-based Latent Variable Models

arXiv: https://arxiv.org/abs/2010.07856 · HTML: https://mlsanity.com/paper/2010.07856/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 15 Oct 2020.

## Built on by (most cited first, 10 of 10)

- [Concrete Score Matching: Generalized Score Matching for Discrete Data](https://mlsanity.com/paper/2211.00802/) (2022)
- [Entropy-based Training Methods for Scalable Neural Implicit Sampler](https://mlsanity.com/paper/2306.04952/) (2023)
- [Score Matching With Missing Data](https://mlsanity.com/paper/2506.00557/) (2025)
- [Bi-level Doubly Variational Learning for Energy-based Latent Variable Models](https://mlsanity.com/paper/2203.14702/) (2022)
- [Learning Proposals for Practical Energy-Based Regression](https://mlsanity.com/paper/2110.11948/) (2021)
- [Conjugate Energy-Based Models](https://mlsanity.com/paper/2106.13798/) (2021)
- [On the failure of variational score matching for VAE models](https://mlsanity.com/paper/2210.13390/) (2022)
- [Self-Adapting Noise-Contrastive Estimation for Energy-Based Models](https://mlsanity.com/paper/2211.02650/) (2022)
- [Denoising Fisher Training For Neural Implicit Samplers](https://mlsanity.com/paper/2411.01453/) (2024)
- [From Data to Rewards: a Bilevel Optimization Perspective on Maximum Likelihood Estimation](https://mlsanity.com/paper/2510.07624/) (2025)

## Abstract (arXiv)

Score matching (SM) provides a compelling approach to learn energy-based models (EBMs) by avoiding the calculation of partition function. However, it remains largely open to learn energy-based latent variable models (EBLVMs), except some special cases. This paper presents a bi-level score matching (BiSM) method to learn EBLVMs with general structures by reformulating SM as a bi-level optimization problem. The higher level introduces a variational posterior of the latent variables and optimizes a modified SM objective, and the lower level optimizes the variational posterior to fit the true posterior. To solve BiSM efficiently, we develop a stochastic optimization algorithm with gradient unrolling. Theoretically, we analyze the consistency of BiSM and the convergence of the stochastic algorithm. Empirically, we show the promise of BiSM in Gaussian restricted Boltzmann machines and highly nonstructural EBLVMs parameterized by deep convolutional neural networks. BiSM is comparable to the widely adopted contrastive divergence and SM methods when they are applicable; and can learn complex EBLVMs with intractable posteriors to generate natural images.