# A Perspective on Neural Capacity Estimation: Viability and Reliability

arXiv: https://arxiv.org/abs/2203.11793 · HTML: https://mlsanity.com/paper/2203.11793/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 17 of its 55 references that are in the map.
- First posted on arXiv 22 Mar 2022.

## Builds on (most cited first, 17 of 17)

- [Representation Learning with Contrastive Predictive Coding](https://mlsanity.com/paper/1807.03748/) (2018)
- [Generative Adversarial Networks](https://mlsanity.com/paper/2203.00667/) (2022)
- [MINE: Mutual Information Neural Estimation](https://mlsanity.com/paper/1801.04062/) (2018)
- [On Variational Bounds of Mutual Information](https://mlsanity.com/paper/1905.06922/) (2019)
- [f-GAN: Training Generative Neural Samplers using Variational Divergence Minimization](https://mlsanity.com/paper/1606.00709/) (2016)
- [Understanding the Limitations of Variational Mutual Information Estimators](https://mlsanity.com/paper/1910.06222/) (2019)
- [CCMI : Classifier based Conditional Mutual Information Estimation](https://mlsanity.com/paper/1906.01824/) (2019)
- [Model-free Training of End-to-end Communication Systems](https://mlsanity.com/paper/1812.05929/) (2018)
- [Neural Estimation of Statistical Divergences](https://mlsanity.com/paper/2110.03652/) (2021)
- [Deep Learning for Channel Coding via Neural Mutual Information Estimation](https://mlsanity.com/paper/1903.02865/) (2019)
- [Joint Learning of Geometric and Probabilistic Constellation Shaping](https://mlsanity.com/paper/1906.07748/) (2019)
- [Neural Entropic Estimation: A faster path to mutual information estimation](https://mlsanity.com/paper/1905.12957/) (2019)
- [Neural Estimators for Conditional Mutual Information Using Nearest Neighbors Sampling](https://mlsanity.com/paper/2006.07225/) (2020)
- [Capacity of Continuous Channels with Memory via Directed Information Neural Estimator](https://mlsanity.com/paper/2003.04179/) (2020)
- [A Reverse Jensen Inequality Result with Application to Mutual Information Estimation](https://mlsanity.com/paper/2111.06676/) (2021)
- [Neural Capacity Estimators: How Reliable Are They?](https://mlsanity.com/paper/2111.07401/) (2021)
- [Computing the Feedback Capacity of Finite State Channels using Reinforcement Learning](https://mlsanity.com/paper/2001.09685/) (2020)

## Abstract (arXiv)

Recently, several methods have been proposed for estimating the mutual information from sample data using deep neural networks. These estimators ar referred to as neural mutual information estimation (NMIE)s. NMIEs differ from other approaches as they are data-driven estimators. As such, they have the potential to perform well on a large class of capacity problems. In order to test the performance across various NMIEs, it is desirable to establish a benchmark encompassing the different challenges of capacity estimation. This is the objective of this paper. In particular, we consider three scenarios for benchmarking:i the classic AWGN channel, ii channels continuous inputs optical intensity and peak-power constrained AWGN channel iii channels with a discrete output, i.e., Poisson channel. We also consider the extension to the multi-terminal case with iv the AWGN and optical MAC models. We argue that benchmarking a certain NMIE across these four scenarios provides a substantive test of performance. In this paper we study the performance of mutual information neural estimator (MINE), smoothed mutual information lower-bound estimator (SMILE), and directed information neural estimator (DINE). and provide insights on the performance of other methods as well. To summarize our benchmarking results, MINE provides the most reliable performance.