# Autoregressive Score Matching

arXiv: https://arxiv.org/abs/2010.12810 · HTML: https://mlsanity.com/paper/2010.12810/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Oct 2020.

## Built on by (most cited first, 15 of 15)

- [Diffusion Models: A Comprehensive Survey of Methods and Applications](https://mlsanity.com/paper/2209.00796/) (2022)
- [Deep Generative Modelling: A Comparative Review of VAEs, GANs, Normalizing Flows, Energy-Based and Autoregressive Models](https://mlsanity.com/paper/2103.04922/) (2021)
- [Learning Gradient Fields for Molecular Conformation Generation](https://mlsanity.com/paper/2105.03902/) (2021)
- [Concrete Score Matching: Generalized Score Matching for Discrete Data](https://mlsanity.com/paper/2211.00802/) (2022)
- [Autoencoding Under Normalization Constraints](https://mlsanity.com/paper/2105.05735/) (2021)
- [Diffusion-based Generative AI for Exploring Transition States from 2D Molecular Graphs](https://mlsanity.com/paper/2304.12233/) (2023)
- [Simulating the Visual World with Artificial Intelligence: A Roadmap](https://mlsanity.com/paper/2511.08585/) (2025)
- [NFIG: Multi-Scale Autoregressive Image Generation via Frequency Ordering](https://mlsanity.com/paper/2503.07076/) (2025)
- [Entropy-based Training Methods for Scalable Neural Implicit Sampler](https://mlsanity.com/paper/2306.04952/) (2023)
- [Training Energy-Based Models with Diffusion Contrastive Divergences](https://mlsanity.com/paper/2307.01668/) (2023)
- [Is Score Matching Suitable for Estimating Point Processes?](https://mlsanity.com/paper/2410.04037/) (2024)
- [TSGM: Regular and Irregular Time-series Generation using Score-based Generative Models](https://mlsanity.com/paper/2511.21335/) (2025)
- [Likelihood Training of Cascaded Diffusion Models via Hierarchical Volume-preserving Maps](https://mlsanity.com/paper/2501.06999/) (2025)
- [Autoregressive Generative Modeling with Noise Conditional Maximum Likelihood Estimation](https://mlsanity.com/paper/2210.10715/) (2022)
- [Score-Based Change-Point Detection and Region Localization for Spatio-Temporal Point Processes](https://mlsanity.com/paper/2602.04798/) (2026)

## Abstract (arXiv)

Autoregressive models use chain rule to define a joint probability distribution as a product of conditionals. These conditionals need to be normalized, imposing constraints on the functional families that can be used. To increase flexibility, we propose autoregressive conditional score models (AR-CSM) where we parameterize the joint distribution in terms of the derivatives of univariate log-conditionals (scores), which need not be normalized. To train AR-CSM, we introduce a new divergence between distributions named Composite Score Matching (CSM). For AR-CSM models, this divergence between data and model distributions can be computed and optimized efficiently, requiring no expensive sampling or adversarial training. Compared to previous score matching algorithms, our method is more scalable to high dimensional data and more stable to optimize. We show with extensive experimental results that it can be applied to density estimation on synthetic data, image generation, image denoising, and training latent variable models with implicit encoders.