# A Stable Neural Statistical Dependence Estimator for Autoencoder Feature Analysis

arXiv: https://arxiv.org/abs/2603.11428 · HTML: https://mlsanity.com/paper/2603.11428/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 7 of its 23 references that are in the map.
- First posted on arXiv 12 Mar 2026.

## Builds on (most cited first, 7 of 7)

- [Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1312.6114/) (2013)
- [InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets](https://mlsanity.com/paper/1606.03657/) (2016)
- [Information Constraints on Auto-Encoding Variational Bayes](https://mlsanity.com/paper/1805.08672/) (2018)
- [An Information-Theoretic Perspective on Variance-Invariance-Covariance Regularization](https://mlsanity.com/paper/2303.00633/) (2023)
- [An Information Theoretic Interpretation to Deep Neural Networks](https://mlsanity.com/paper/1905.06600/) (2019)
- [Neural Feature Learning in Function Space](https://mlsanity.com/paper/2309.10140/) (2023)
- [Learning Cortico-Muscular Dependence through Orthonormal Decomposition of Density Ratios](https://mlsanity.com/paper/2410.14697/) (2024)

## Abstract (arXiv)

Statistical dependence measures like mutual information is ideal for analyzing autoencoders, but it can be ill-posed for deterministic, static, noise-free networks. We adopt the variational (Gaussian) formulation that makes dependence among inputs, latents, and reconstructions measurable, and we propose a stable neural dependence estimator based on an orthonormal density-ratio decomposition. Unlike MINE, our method avoids input concatenation and product-of-marginals re-pairing, reducing computational cost and improving stability. We introduce an efficient NMF-like scalar cost and demonstrate empirically that assuming Gaussian noise to form an auxiliary variable enables meaningful dependence measurements and supports quantitative feature analysis, with a sequential convergence of singular values.