# InfoAtlas: A Foundation Model for Zero-Shot Statistical Dependence Estimate

arXiv: https://arxiv.org/abs/2606.00241 · HTML: https://mlsanity.com/paper/2606.00241/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 40 of its 60 references that are in the map.
- First posted on arXiv 29 May 2026.

## Builds on (most cited first, 30 of 40)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities](https://mlsanity.com/paper/2507.06261/) (2025)
- [Density estimation using Real NVP](https://mlsanity.com/paper/1605.08803/) (2016)
- [Microsoft COCO Captions: Data Collection and Evaluation Server](https://mlsanity.com/paper/1504.00325/) (2015)
- [Normalizing Flows for Probabilistic Modeling and Inference](https://mlsanity.com/paper/1912.02762/) (2019)
- [InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets](https://mlsanity.com/paper/1606.03657/) (2016)
- [Chronos: Learning the Language of Time Series](https://mlsanity.com/paper/2403.07815/) (2024)
- [MINE: Mutual Information Neural Estimation](https://mlsanity.com/paper/1801.04062/) (2018)
- [Perceiver IO: A General Architecture for Structured Inputs & Outputs](https://mlsanity.com/paper/2107.14795/) (2021)
- [On Variational Bounds of Mutual Information](https://mlsanity.com/paper/1905.06922/) (2019)
- [ManiSkill2: A Unified Benchmark for Generalizable Manipulation Skills](https://mlsanity.com/paper/2302.04659/) (2023)
- [On Mutual Information Maximization for Representation Learning](https://mlsanity.com/paper/1907.13625/) (2019)
- [PointOdyssey: A Large-Scale Synthetic Dataset for Long-Term Point Tracking](https://mlsanity.com/paper/2307.15055/) (2023)
- [Fast Differentiable Sorting and Ranking](https://mlsanity.com/paper/2002.08871/) (2020)
- [Formal Limitations on the Measurement of Mutual Information](https://mlsanity.com/paper/1811.04251/) (2018)
- [Understanding the Limitations of Variational Mutual Information Estimators](https://mlsanity.com/paper/1910.06222/) (2019)
- [Neural Approximate Sufficient Statistics for Implicit Models](https://mlsanity.com/paper/2010.10079/) (2020)
- [LLM Processes: Numerical Predictive Distributions Conditioned on Natural Language](https://mlsanity.com/paper/2405.12856/) (2024)
- [Beyond Normal: On the Evaluation of Mutual Information Estimators](https://mlsanity.com/paper/2306.11078/) (2023)
- [MINDE: Mutual Information Neural Diffusion Estimation](https://mlsanity.com/paper/2310.09031/) (2023)
- [LAMBDA: A Large Model Based Data Agent](https://mlsanity.com/paper/2407.17535/) (2024)
- [Neural Methods for Point-wise Dependency Estimation](https://mlsanity.com/paper/2006.05553/) (2020)
- [Mutual Information Estimation via Normalizing Flows](https://mlsanity.com/paper/2403.02187/) (2024)
- [Approximating mutual information of high-dimensional variables using learned representations](https://mlsanity.com/paper/2409.02732/) (2024)
- [A Differential Entropy Estimator for Training Neural Networks](https://mlsanity.com/paper/2202.06618/) (2022)
- [Information Bottleneck Analysis of Deep Neural Networks via Lossy Compression](https://mlsanity.com/paper/2305.08013/) (2023)
- [Mutual Information Estimation via f-Divergence and Data Derangements](https://mlsanity.com/paper/2305.20025/) (2023)
- [Effortless, Simulation-Efficient Bayesian Inference using Tabular Foundation Models](https://mlsanity.com/paper/2504.17660/) (2025)

## Built on by (most cited first, 1 of 1)

- [ALICE: In-context, Zero-shot, Mutual Information Estimation](https://mlsanity.com/paper/2609.34962/) (2026)

## Abstract (arXiv)

Measuring statistical dependency between high-dimensional random variables is a fundamental task in data science and machine learning. Neural mutual information (MI) estimators offer a promising avenue, but they typically require costly iterative optimization for each new dataset, making them impractical for real-time applications. We present InfoAtlas, a foundation model-like architecture that eliminates this bottleneck by directly inferring MI in a single forward pass. Pretrained on large-scale synthetic data with rich dependence patterns, InfoAtlas learns to identify diverse dependence structures and predict MI directly from the dataset. Comprehensive experiments demonstrate that InfoAtlas matches state-of-the-art neural estimators in accuracy while achieving 100× speedup, can flexibly handle varying dimensions and sample sizes through a single unified model, and generalizes effectively to complex, real-world scenarios. By reformulating MI estimation as an inference task, InfoAtlas establishes a foundation for real-time dependency analysis.