# Distributional Autoencoders Know the Score

arXiv: https://arxiv.org/abs/2502.11583 · HTML: https://mlsanity.com/paper/2502.11583/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 28 references that are in the map.
- First posted on arXiv 17 Feb 2025.

## Builds on (most cited first, 12 of 12)

- [Score-Based Generative Modeling through Stochastic Differential Equations](https://mlsanity.com/paper/2011.13456/) (2020)
- [Disentangling by Factorising](https://mlsanity.com/paper/1802.05983/) (2018)
- [VAMPnets: Deep learning of molecular kinetics](https://mlsanity.com/paper/1710.06012/) (2017)
- [Two for One: Diffusion Models and Force Fields for Coarse-Grained Molecular Dynamics](https://mlsanity.com/paper/2302.00600/) (2023)
- [Engression: Extrapolation through the Lens of Distributional Regression](https://mlsanity.com/paper/2307.00835/) (2023)
- [Chart Auto-Encoders for Manifold Structured Data](https://mlsanity.com/paper/1912.10094/) (2019)
- [Distributional Diffusion Models with Scoring Rules](https://mlsanity.com/paper/2502.02483/) (2025)
- [Molecular enhanced sampling with autoencoders: On-the-fly collective variable discovery and accelerated free energy landscape exploration](https://mlsanity.com/paper/1801.00203/) (2017)
- [Learning Manifold Dimensions with Conditional Variational Autoencoders](https://mlsanity.com/paper/2302.11756/) (2023)
- [Distributional Principal Autoencoders](https://mlsanity.com/paper/2404.13649/) (2024)
- [Deep Nonparametric Estimation of Intrinsic Data Structures by Chart Autoencoders: Generalization Error and Robustness](https://mlsanity.com/paper/2303.09863/) (2023)
- [Convergent autoencoder approximation of low bending and low distortion manifold embeddings](https://mlsanity.com/paper/2208.10193/) (2022)

## Built on by (most cited first, 3 of 3)

- [Energy-Tweedie: Score meets Score, Energy meets Energy](https://mlsanity.com/paper/2512.23818/) (2025)
- [Bilateral Distribution Compression: Reducing Both Data Size and Dimensionality](https://mlsanity.com/paper/2509.17543/) (2025)
- [Conditional-Independence-Regularized Distributional Autoencoders for Mixed-Type Data](https://mlsanity.com/paper/2608.20562/) (2026)

## Abstract (arXiv)

The Distributional Principal Autoencoder (DPA) combines distributionally correct reconstruction with principal-component-like interpretability of the encodings. In this work, we provide exact theoretical guarantees on both fronts. First, we derive a closed-form relation linking each optimal level-set geometry to the data-distribution score. This result explains DPA's empirical ability to disentangle factors of variation of the data, as well as allows the score to be recovered directly from samples. When the data follows the Boltzmann distribution, we demonstrate that this relation yields an approximation of the minimum free-energy path for the Mueller-Brown potential in a single fit. Second, we prove that if the data lies on a manifold that can be approximated by the encoder, latent components beyond the manifold dimension are conditionally independent of the data distribution - carrying no additional information - and thus reveal the intrinsic dimension. Together, these results show that a single model can learn the data distribution and its intrinsic dimension with exact guarantees simultaneously, unifying two longstanding goals of unsupervised learning.