# Data-Driven Estimation of Capacity Upper Bounds

arXiv: https://arxiv.org/abs/2205.06471 · HTML: https://mlsanity.com/paper/2205.06471/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 11 of its 31 references that are in the map.
- First posted on arXiv 13 May 2022.

## Builds on (most cited first, 11 of 11)

- [Adam: A Method for Stochastic Optimization](https://mlsanity.com/paper/1412.6980/) (2014)
- [MINE: Mutual Information Neural Estimation](https://mlsanity.com/paper/1801.04062/) (2018)
- [On Variational Bounds of Mutual Information](https://mlsanity.com/paper/1905.06922/) (2019)
- [An Introduction to Deep Learning for the Physical Layer](https://mlsanity.com/paper/1702.00832/) (2017)
- [Formal Limitations on the Measurement of Mutual Information](https://mlsanity.com/paper/1811.04251/) (2018)
- [Learning Deep Embeddings with Histogram Loss](https://mlsanity.com/paper/1611.00822/) (2016)
- [Deep Learning for Channel Coding via Neural Mutual Information Estimation](https://mlsanity.com/paper/1903.02865/) (2019)
- [Achievable Information Rates for Nonlinear Fiber Communication via End-to-end Autoencoder Learning](https://mlsanity.com/paper/1804.07675/) (2018)
- [Capacity of Continuous Channels with Memory via Directed Information Neural Estimator](https://mlsanity.com/paper/2003.04179/) (2020)
- [Neural Capacity Estimators: How Reliable Are They?](https://mlsanity.com/paper/2111.07401/) (2021)
- [Computing the Feedback Capacity of Finite State Channels using Reinforcement Learning](https://mlsanity.com/paper/2001.09685/) (2020)

## Abstract (arXiv)

We consider the problem of estimating an upper bound on the capacity of a memoryless channel with unknown channel law and continuous output alphabet. A novel data-driven algorithm is proposed that exploits the dual representation of capacity where the maximization over the input distribution is replaced with a minimization over a reference distribution on the channel output. To efficiently compute the required divergence maximization between the conditional channel and the reference distribution, we use a modified mutual information neural estimator that takes the channel input as an additional parameter. We numerically evaluate our approach on different memoryless channels and show empirically that the estimated upper bounds closely converge either to the channel capacity or to best-known lower bounds.