# Learning to Embed Distributions via Maximum Kernel Entropy

arXiv: https://arxiv.org/abs/2408.00549 · HTML: https://mlsanity.com/paper/2408.00549/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 71 references that are in the map.
- First posted on arXiv 1 Aug 2024.

## Builds on (most cited first, 30 of 30)

- [Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms](https://mlsanity.com/paper/1708.07747/) (2017)
- [Understanding Contrastive Representation Learning through Alignment and Uniformity on the Hypersphere](https://mlsanity.com/paper/2005.10242/) (2020)
- [Domain Generalization via Invariant Feature Representation](https://mlsanity.com/paper/1301.2115/) (2013)
- [Kernel Mean Embedding of Distributions: A Review and Beyond](https://mlsanity.com/paper/1605.09522/) (2016)
- [Random Features for Kernel Approximation: A Survey on Algorithms, Theory, and Beyond](https://mlsanity.com/paper/2004.11154/) (2020)
- [Kernel Operations on the GPU, with Autodiff, without Memory Overflows](https://mlsanity.com/paper/2004.11127/) (2020)
- [Less is More: Nystr"om Computational Regularization](https://mlsanity.com/paper/1507.04717/) (2015)
- [Kernel methods through the roof: handling billions of points efficiently](https://mlsanity.com/paper/2006.10350/) (2020)
- [Learning Deep Features in Instrumental Variable Regression](https://mlsanity.com/paper/2010.07154/) (2020)
- [Sliced Wasserstein Kernels for Probability Distributions](https://mlsanity.com/paper/1511.03198/) (2015)
- [Learning Theory for Distribution Regression](https://mlsanity.com/paper/1411.2066/) (2014)
- [Kernel Distribution Embeddings: Universal Kernels, Characteristic Kernels and Kernel Metrics on Distributions](https://mlsanity.com/paper/1604.05251/) (2016)
- [Deep Proxy Causal Learning and its Application to Confounded Bandit Policy Evaluation](https://mlsanity.com/paper/2106.03907/) (2021)
- [Reproducing Kernel Hilbert Space, Mercer's Theorem, Eigenfunctions, Nystr"om Method, and Use of Kernels in Machine Learning: Tutorial and Survey](https://mlsanity.com/paper/2106.08443/) (2021)
- [Information Theory with Kernel Methods](https://mlsanity.com/paper/2202.08545/) (2022)
- [Distribution Regression for Sequential Data](https://mlsanity.com/paper/2006.05805/) (2020)
- [Distribution-Free Distribution Regression](https://mlsanity.com/paper/1302.0082/) (2013)
- [Sliced-Wasserstein on Symmetric Positive Definite Matrices for M/EEG Signals](https://mlsanity.com/paper/2303.05798/) (2023)
- [Distribution Regression with Sliced Wasserstein Kernels](https://mlsanity.com/paper/2202.03926/) (2022)
- [VNE: An Effective Method for Improving Deep Representation by Manipulating Eigenvalue Distribution](https://mlsanity.com/paper/2304.01434/) (2023)
- [Meta Learning for Causal Direction](https://mlsanity.com/paper/2007.02809/) (2020)
- [Information Flow in Self-Supervised Learning](https://mlsanity.com/paper/2309.17281/) (2023)
- [Generalizing and Decoupling Neural Collapse via Hyperspherical Uniformity Gap](https://mlsanity.com/paper/2303.06484/) (2023)
- [A Gaussian Process Regression Model for Distribution Inputs](https://mlsanity.com/paper/1701.09055/) (2017)
- [Noise Contrastive Meta-Learning for Conditional Density Estimation using Kernel Mean Embeddings](https://mlsanity.com/paper/1906.02236/) (2019)
- [Bayesian Approaches to Distribution Regression](https://mlsanity.com/paper/1705.04293/) (2017)
- [Robust Kernel-based Distribution Regression](https://mlsanity.com/paper/2104.10637/) (2021)
- [DR-ABC: Approximate Bayesian Computation with Kernel-Based Distribution Regression](https://mlsanity.com/paper/1602.04805/) (2016)
- [Understanding the 2016 US Presidential Election using ecological inference and distribution regression with census microdata](https://mlsanity.com/paper/1611.03787/) (2016)
- [Distance Measure Machines](https://mlsanity.com/paper/1803.00250/) (2018)

## Built on by (most cited first, 3 of 3)

- [Statistical Learning Theory for Distributional Classification](https://mlsanity.com/paper/2601.14818/) (2026)
- [Quantum Maximum Likelihood Prediction via Hilbert Space Embeddings](https://mlsanity.com/paper/2602.18364/) (2026)
- [Semi-Supervised Learning with Noisy Proxy Covariates: Generalization Bounds and Distribution Regression](https://mlsanity.com/paper/2606.00512/) (2026)

## Abstract (arXiv)

Empirical data can often be considered as samples from a set of probability distributions. Kernel methods have emerged as a natural approach for learning to classify these distributions. Although numerous kernels between distributions have been proposed, applying kernel methods to distribution regression tasks remains challenging, primarily because selecting a suitable kernel is not straightforward. Surprisingly, the question of learning a data-dependent distribution kernel has received little attention. In this paper, we propose a novel objective for the unsupervised learning of data-dependent distribution kernel, based on the principle of entropy maximization in the space of probability measure embeddings. We examine the theoretical properties of the latent embedding space induced by our objective, demonstrating that its geometric structure is well-suited for solving downstream discriminative tasks. Finally, we demonstrate the performance of the learned kernel across different modalities.