# Information Bottleneck Meets Quantization: Finite Rate Analysis and Optimal Designs

arXiv: https://arxiv.org/abs/2606.10869 · HTML: https://mlsanity.com/paper/2606.10869/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 21 of its 40 references that are in the map.
- First posted on arXiv 9 Jun 2026.

## Builds on (most cited first, 21 of 21)

- [Neural Discrete Representation Learning](https://mlsanity.com/paper/1711.00937/) (2017)
- [Convex optimization](https://mlsanity.com/paper/2106.01946/) (2021)
- [Deep Variational Information Bottleneck](https://mlsanity.com/paper/1612.00410/) (2016)
- [Beyond Transmitting Bits: Context, Semantics, and Task-Oriented Communications](https://mlsanity.com/paper/2207.09353/) (2022)
- [6G Networks: Beyond Shannon Towards Semantic and Goal-Oriented Communications](https://mlsanity.com/paper/2011.14844/) (2020)
- [Learning Task-Oriented Communication for Edge Inference: An Information Bottleneck Approach](https://mlsanity.com/paper/2102.04170/) (2021)
- [Robust Semantic Communications with Masked VQ-VAE Enabled Codebook](https://mlsanity.com/paper/2206.04011/) (2022)
- [Task-Oriented Communication for Multi-Device Cooperative Edge Inference](https://mlsanity.com/paper/2109.00172/) (2021)
- [Robust Information Bottleneck for Task-Oriented Communication with Digital Modulation](https://mlsanity.com/paper/2209.10382/) (2022)
- [Learnability for the Information Bottleneck](https://mlsanity.com/paper/1907.07331/) (2019)
- [D2-JSCC: Digital Deep Joint Source-channel Coding for Semantic Communications](https://mlsanity.com/paper/2403.07338/) (2024)
- [Goal-oriented Communications for the IoT: System Design and Adaptive Resource Optimization](https://mlsanity.com/paper/2310.13948/) (2023)
- [OFDM-Based Digital Semantic Communication with Importance Awareness](https://mlsanity.com/paper/2401.02178/) (2024)
- [Towards Human-Agent Communication via the Information Bottleneck Principle](https://mlsanity.com/paper/2207.00088/) (2022)
- [Adaptive Information Bottleneck Guided Joint Source and Channel Coding for Image Transmission](https://mlsanity.com/paper/2203.06492/) (2022)
- [Task-Driven Semantic Quantization and Imitation Learning for Goal-Oriented Communications](https://mlsanity.com/paper/2502.17842/) (2025)
- [Fast Sequence Segmentation using Log-Linear Models](https://mlsanity.com/paper/1902.03285/) (2019)
- [Opportunistic Information-Bottleneck for Goal-oriented Feature Extraction and Communication](https://mlsanity.com/paper/2404.09218/) (2024)
- [Channel-Aware Vector Quantization for Robust Semantic Communication on Discrete Channels](https://mlsanity.com/paper/2510.18604/) (2025)
- [Scalable Vector Gaussian Information Bottleneck](https://mlsanity.com/paper/2102.07525/) (2021)
- [VQ-VAE Based Digital Semantic Communication with Importance-Aware OFDM Transmission](https://mlsanity.com/paper/2508.08686/) (2025)

## Abstract (arXiv)

The Information Bottleneck (IB) is a well established framework that looks for a latent compact representation of a data source, by trading rate and data-size representation, for information accuracy with respect to another target data. The Gaussian IB (GIB) is its simple closed form solution, when the target is jointly Gaussian with the source. Actually, in many practical problems the latent representation has to be stored or represented by a finite number of bits, while the optimal (G)IB solution has not. First, this manuscript theoretically analyzes the effect of scalar and vector quantization of the GIB latent representation, and its impact on the (dis)informativeness with respect to the target data. Then, task-oriented quantization designs are proposed by (jointly) reformulating the GIB optimization problem under a finite-rate constraint on the latent representation. Simulation results on MMSE regression problems confirm the effectiveness of the proposed quantization designs, which show significant gains with respect to more heuristic, or separate, quantization designs of the standard GIB latent representation. Finally, the paper extends the task-oriented philosophy to non-Gaussian settings, by properly modifying the cost function used in variational auto-encoders (VAEs) of IB-inspired vector quantizers.