# On the Cardinality of Optimal Representations in the Binary-Source Information Bottleneck

arXiv: https://arxiv.org/abs/2610.06627 · HTML: https://mlsanity.com/paper/2610.06627/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 2 of its 13 references that are in the map.
- First posted on arXiv 5 Oct 2026.

## Builds on (most cited first, 2 of 2)

- [Opening the Black Box of Deep Neural Networks via Information](https://mlsanity.com/paper/1703.00810/) (2017)
- [Mutual Information Constrained Chernoff Bottleneck](https://mlsanity.com/paper/2609.37994/) (2026)

## Abstract (arXiv)

The information bottleneck (IB) seeks a representation U of a source X that retains as much information as possible about a target Y, subject to a constraint on I(U;X). A classical argument shows that it suffices to consider representations with at most |X|+1 symbols, and this bound is known to be tight whenever |X| ≥ 3. We show that the binary case behaves differently: if X is binary and Y is finite, then for every joint distribution of (X,Y) and every rate constraint, the IB optimum is attained by a binary U. Hence the bound |U| ≤ |X|+1 sharpens to |U| ≤ |X| for binary sources. The proof combines a separating hyperplane argument with the observation that, for a binary source, the ratio of the second derivatives of the two entropy functions involved is concave.