# Joint Stochastic Approximation and Its Application to Learning Discrete Latent Variable Models

arXiv: https://arxiv.org/abs/2005.14001 · HTML: https://mlsanity.com/paper/2005.14001/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 28 May 2020.

## Built on by (most cited first, 10 of 10)

- [Coupled Gradient Estimators for Discrete Latent Variables](https://mlsanity.com/paper/2106.08056/) (2021)
- [Transport Score Climbing: Variational Inference Using Forward KL and Adaptive Neural Transport](https://mlsanity.com/paper/2202.01841/) (2022)
- [Knowledge-Retrieval Task-Oriented Dialog Systems with Semi-Supervision](https://mlsanity.com/paper/2305.13199/) (2023)
- [Markov Chain Score Ascent: A Unifying Framework of Variational Inference with Markovian Gradients](https://mlsanity.com/paper/2206.06295/) (2022)
- [Disentangling impact of capacity, objective, batchsize, estimators, and step-size on flow VI](https://mlsanity.com/paper/2412.08824/) (2024)
- [Advancing Semi-Supervised Task Oriented Dialog Systems by JSA Learning of Discrete Latent Variable Models](https://mlsanity.com/paper/2207.12235/) (2022)
- [Pronunciation-Lexicon Free Training for Phoneme-based Crosslingual ASR via Joint Stochastic Approximation](https://mlsanity.com/paper/2507.06249/) (2025)
- [Improving End-to-End Training of Retrieval-Augmented Generation Models via Joint Stochastic Approximation](https://mlsanity.com/paper/2508.18168/) (2025)
- [Variational Inference for Neyman-Scott Processes](https://mlsanity.com/paper/2303.03701/) (2023)
- [Entriever: Energy-based Retriever for Knowledge-Grounded Dialog Systems](https://mlsanity.com/paper/2506.00585/) (2025)

## Abstract (arXiv)

Although with progress in introducing auxiliary amortized inference models, learning discrete latent variable models is still challenging. In this paper, we show that the annoying difficulty of obtaining reliable stochastic gradients for the inference model and the drawback of indirectly optimizing the target log-likelihood can be gracefully addressed in a new method based on stochastic approximation (SA) theory of the Robbins-Monro type. Specifically, we propose to directly maximize the target log-likelihood and simultaneously minimize the inclusive divergence between the posterior and the inference model. The resulting learning algorithm is called joint SA (JSA). To the best of our knowledge, JSA represents the first method that couples an SA version of the EM (expectation-maximization) algorithm (SAEM) with an adaptive MCMC procedure. Experiments on several benchmark generative modeling and structured prediction tasks show that JSA consistently outperforms recent competitive algorithms, with faster convergence, better final likelihoods, and lower variance of gradient estimates.