# Learning deep kernels for exponential family densities

arXiv: https://arxiv.org/abs/1811.08357 · HTML: https://mlsanity.com/paper/1811.08357/

## Key facts

- Cited by 36 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Nov 2018.

## Built on by (most cited first, 30 of 36)

- [How to Train Your Energy-Based Models](https://mlsanity.com/paper/2101.03288/) (2021)
- [Robust Generalised Bayesian Inference for Intractable Likelihoods](https://mlsanity.com/paper/2104.07359/) (2021)
- [ScoreGrad: Multivariate Probabilistic Time Series Forecasting with Continuous Energy-based Generative Models](https://mlsanity.com/paper/2106.10121/) (2021)
- [Subtractive Mixture Models via Squaring: Representation and Learning](https://mlsanity.com/paper/2310.00724/) (2023)
- [End-to-End Stochastic Optimization with Energy-Based Model](https://mlsanity.com/paper/2211.13837/) (2022)
- [Composite Goodness-of-fit Tests with Kernels](https://mlsanity.com/paper/2111.10275/) (2021)
- [Efficient Training of Energy-Based Models Using Jarzynski Equality](https://mlsanity.com/paper/2305.19414/) (2023)
- [Towards Healing the Blindness of Score Matching](https://mlsanity.com/paper/2209.07396/) (2022)
- [CADet: Fully Self-Supervised Out-Of-Distribution Detection With Contrastive Learning](https://mlsanity.com/paper/2210.01742/) (2022)
- [A Computationally Efficient Method for Learning Exponential Family Distributions](https://mlsanity.com/paper/2110.15397/) (2021)
- [Sampling Multimodal Distributions with the Vanilla Score: Benefits of Data-Based Initialization](https://mlsanity.com/paper/2310.01762/) (2023)
- [Interpreting diffusion score matching using normalizing flow](https://mlsanity.com/paper/2107.10072/) (2021)
- [The Representation Jensen-Shannon Divergence](https://mlsanity.com/paper/2305.16446/) (2023)
- [CFMI: Flow Matching for Missing Data Imputation](https://mlsanity.com/paper/2506.09258/) (2025)
- [Highly Accurate Real-space Electron Densities with Neural Networks](https://mlsanity.com/paper/2409.01306/) (2024)
- [Active Slices for Sliced Stein Discrepancy](https://mlsanity.com/paper/2102.03159/) (2021)
- [EBM-Fold: Fully-Differentiable Protein Folding Powered by Energy-based Models](https://mlsanity.com/paper/2105.04771/) (2021)
- [Semantic Driven Energy based Out-of-Distribution Detection](https://mlsanity.com/paper/2208.10787/) (2022)
- [On the failure of variational score matching for VAE models](https://mlsanity.com/paper/2210.13390/) (2022)
- [On Computationally Efficient Learning of Exponential Family Distributions](https://mlsanity.com/paper/2309.06413/) (2023)
- [ISR: Invertible Symbolic Regression](https://mlsanity.com/paper/2405.06848/) (2024)
- [Denoising Score Matching with Random Fourier Features](https://mlsanity.com/paper/2101.05239/) (2021)
- [A Kernelised Stein Statistic for Assessing Implicit Generative Models](https://mlsanity.com/paper/2206.00149/) (2022)
- [Consecutive Knowledge Meta-Adaptation Learning for Unsupervised Medical Diagnosis](https://mlsanity.com/paper/2209.10425/) (2022)
- [Incorporating Inductive Biases to Energy-based Generative Models](https://mlsanity.com/paper/2505.01111/) (2025)
- [Kernel Deformed Exponential Families for Sparse Continuous Attention](https://mlsanity.com/paper/2111.01222/) (2021)
- [Deep Functional Factor Models: Forecasting High-Dimensional Functional Time Series via Bayesian Nonparametric Factorization](https://mlsanity.com/paper/2305.14543/) (2023)
- [Kernelised Normalising Flows](https://mlsanity.com/paper/2307.14839/) (2023)
- [STANLEY: Stochastic Gradient Anisotropic Langevin Dynamics for Learning Energy-Based Models](https://mlsanity.com/paper/2310.12667/) (2023)
- [Score Change of Variables](https://mlsanity.com/paper/2412.07904/) (2024)

## Abstract (arXiv)

The kernel exponential family is a rich class of distributions, which can be fit efficiently and with statistical guarantees by score matching. Being required to choose a priori a simple kernel such as the Gaussian, however, limits its practical applicability. We provide a scheme for learning a kernel parameterized by a deep network, which can find complex location-dependent local features of the data geometry. This gives a very rich class of density models, capable of fitting complex structures on moderate-dimensional problems. Compared to deep density models fit via maximum likelihood, our approach provides a complementary set of strengths and tradeoffs: in empirical studies, the former can yield higher likelihoods, whereas the latter gives better estimates of the gradient of the log density, the score, which describes the distribution's shape.