# Bottleneck Problems: Information and Estimation-Theoretic View

arXiv: https://arxiv.org/abs/2011.06208 · HTML: https://mlsanity.com/paper/2011.06208/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 12 Nov 2020.

## Built on by (most cited first, 10 of 10)

- [Bottlenecks CLUB: Unifying Information-Theoretic Trade-offs Among Complexity, Leakage, and Utility](https://mlsanity.com/paper/2207.04895/) (2022)
- [Interpretability with full complexity by constraining feature information](https://mlsanity.com/paper/2211.17264/) (2022)
- [Lossless Transformations and Excess Risk Bounds in Statistical Inference](https://mlsanity.com/paper/2307.16735/) (2023)
- [The Fundamental Limits of Least-Privilege Learning](https://mlsanity.com/paper/2402.12235/) (2024)
- [An Efficient Difference-of-Convex Solver for Privacy Funnel](https://mlsanity.com/paper/2403.04778/) (2024)
- [Classification Utility, Fairness, and Compactness via Tunable Information Bottleneck and R'enyi Measures](https://mlsanity.com/paper/2206.10043/) (2022)
- [Deep Privacy Funnel Model: From a Discriminative to a Generative Approach with an Application to Face Recognition](https://mlsanity.com/paper/2404.02696/) (2024)
- [Source-Side Sufficiency for the Information Bottleneck: Exact Reduction and Finite-Block Equivalence](https://mlsanity.com/paper/2604.26744/) (2026)
- [Characterizing information loss in a chaotic double pendulum with the Information Bottleneck](https://mlsanity.com/paper/2210.14220/) (2022)
- [Information Bottleneck under Perfect Privacy](https://mlsanity.com/paper/2608.11003/) (2026)

## Abstract (arXiv)

Information bottleneck (IB) and privacy funnel (PF) are two closely related optimization problems which have found applications in machine learning, design of privacy algorithms, capacity problems (e.g., Mrs. Gerber's Lemma), strong data processing inequalities, among others. In this work, we first investigate the functional properties of IB and PF through a unified theoretical framework. We then connect them to three information-theoretic coding problems, namely hypothesis testing against independence, noisy source coding and dependence dilution. Leveraging these connections, we prove a new cardinality bound for the auxiliary variable in IB, making its computation more tractable for discrete random variables. In the second part, we introduce a general family of optimization problems, termed as textitbottleneck problems, by replacing mutual information in IB and PF with other notions of mutual information, namely f-information and Arimoto's mutual information. We then argue that, unlike IB and PF, these problems lead to easily interpretable guarantee in a variety of inference tasks with statistical constraints on accuracy and privacy. Although the underlying optimization problems are non-convex, we develop a technique to evaluate bottleneck problems in closed form by equivalently expressing them in terms of lower convex or upper concave envelope of certain functions. By applying this technique to binary case, we derive closed form expressions for several bottleneck problems.