# Information Theoretical Estimators Toolbox

arXiv: https://arxiv.org/abs/1405.2106 · HTML: https://mlsanity.com/paper/1405.2106/

## Key facts

- Cited by 15 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 8 May 2014.

## Built on by (most cited first, 15 of 15)

- [Provable Benefit of Annealed Langevin Monte Carlo for Non-log-concave Sampling](https://mlsanity.com/paper/2407.16936/) (2024)
- [Semi-Implicit Variational Inference via Score Matching](https://mlsanity.com/paper/2308.10014/) (2023)
- [Reflected Flow Matching](https://mlsanity.com/paper/2405.16577/) (2024)
- [Hierarchical Semi-Implicit Variational Inference with Application to Diffusion Model Acceleration](https://mlsanity.com/paper/2310.17153/) (2023)
- [Looks Too Good To Be True: An Information-Theoretic Analysis of Hallucinations in Generative Restoration Models](https://mlsanity.com/paper/2405.16475/) (2024)
- [Kernel Semi-Implicit Variational Inference](https://mlsanity.com/paper/2405.18997/) (2024)
- [Efficient Multi-Modal Embeddings from Structured Data](https://mlsanity.com/paper/2110.02577/) (2021)
- [Information-Theoretic Measures on Lattices for High-Order Interactions](https://mlsanity.com/paper/2408.07533/) (2024)
- [Picking Up Quantization Steps for Compressed Image Classification](https://mlsanity.com/paper/2304.10714/) (2023)
- [Learning Representations for Independence Testing](https://mlsanity.com/paper/2409.06890/) (2024)
- [Enabling Inference Privacy with Adaptive Noise Injection](https://mlsanity.com/paper/2104.02261/) (2021)
- [Causal Identification with Additive Noise Models: Quantifying the Effect of Noise](https://mlsanity.com/paper/2110.08087/) (2021)
- [Bayesian Adaptive Calibration and Optimal Design](https://mlsanity.com/paper/2405.14440/) (2024)
- [A Kernel Approach for Semi-implicit Variational Inference](https://mlsanity.com/paper/2601.12023/) (2026)
- [The Effect of Noise Level on Causal Identification with Additive Noise Models](https://mlsanity.com/paper/2108.11320/) (2021)

## Abstract (arXiv)

We present ITE (information theoretical estimators) a free and open source, multi-platform, Matlab/Octave toolbox that is capable of estimating many different variants of entropy, mutual information, divergence, association measures, cross quantities, and kernels on distributions. Thanks to its highly modular design, ITE supports additionally (i) the combinations of the estimation techniques, (ii) the easy construction and embedding of novel information theoretical estimators, and (iii) their immediate application in information theoretical optimization problems. ITE also includes a prototype application in a central problem class of signal processing, independent subspace analysis and its extensions.