# Minimum Entropy Coupling with Bottleneck

arXiv: https://arxiv.org/abs/2410.21666 · HTML: https://mlsanity.com/paper/2410.21666/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 45 references that are in the map.
- First posted on arXiv 29 Oct 2024.

## Builds on (most cited first, 17 of 17)

- [Image-to-Image Translation with Conditional Adversarial Networks](https://mlsanity.com/paper/1611.07004/) (2016)
- [Learning deep representations by mutual information estimation and maximization](https://mlsanity.com/paper/1808.06670/) (2018)
- [CyCADA: Cycle-Consistent Adversarial Domain Adaptation](https://mlsanity.com/paper/1711.03213/) (2017)
- [InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets](https://mlsanity.com/paper/1606.03657/) (2016)
- [A Watermark for Large Language Models](https://mlsanity.com/paper/2301.10226/) (2023)
- [On Mutual Information Maximization for Representation Learning](https://mlsanity.com/paper/1907.13625/) (2019)
- [Rethinking Lossy Compression: The Rate-Distortion-Perception Tradeoff](https://mlsanity.com/paper/1901.07821/) (2019)
- [Perfectly Secure Steganography Using Minimum Entropy Coupling](https://mlsanity.com/paper/2210.14889/) (2022)
- [Entropic Causal Inference](https://mlsanity.com/paper/1611.04035/) (2016)
- [Entropic Causal Inference: Identifiability and Finite Sample Results](https://mlsanity.com/paper/2101.03501/) (2021)
- [Communicating via Markov Decision Processes](https://mlsanity.com/paper/2107.08295/) (2021)
- [Minimum--Entropy Couplings and their Applications](https://mlsanity.com/paper/1901.07530/) (2019)
- [Minimum-Entropy Coupling Approximation Guarantees Beyond the Majorization Barrier](https://mlsanity.com/paper/2302.11838/) (2023)
- [How to Find a Joint Probability Distribution of Minimum Entropy (almost) given the Marginals](https://mlsanity.com/paper/1701.05243/) (2017)
- [Asymptotic Coupling and Its Applications in Information Theory](https://mlsanity.com/paper/1712.06804/) (2017)
- [Toward Joint Image Generation and Compression using Generative Adversarial Networks](https://mlsanity.com/paper/1901.07838/) (2019)
- [A Tighter Approximation Guarantee for Greedy Minimum Entropy Coupling](https://mlsanity.com/paper/2203.05108/) (2022)

## Built on by (most cited first, 5 of 5)

- [Learning to Match Unpaired Data with Minimum Entropy Coupling](https://mlsanity.com/paper/2503.08501/) (2025)
- [From Exposure to Internalization: Dual-Stream Calibration for In-context Clinical Reasoning](https://mlsanity.com/paper/2604.06262/) (2026)
- [Cross-Domain Lossy Compression via Constrained Minimum Entropy Coupling](https://mlsanity.com/paper/2605.09833/) (2026)
- [Learning from Totally Unpaired Data via Minimum Conditional Entropy Coupling](https://mlsanity.com/paper/2605.16229/) (2026)
- [Structural Limits of the Information-Theoretic Uncertainty Decomposition](https://mlsanity.com/paper/2609.39591/) (2026)

## Abstract (arXiv)

This paper investigates a novel lossy compression framework operating under logarithmic loss, designed to handle situations where the reconstruction distribution diverges from the source distribution. This framework is especially relevant for applications that require joint compression and retrieval, and in scenarios involving distributional shifts due to processing. We show that the proposed formulation extends the classical minimum entropy coupling framework by integrating a bottleneck, allowing for a controlled degree of stochasticity in the coupling. We explore the decomposition of the Minimum Entropy Coupling with Bottleneck (MEC-B) into two distinct optimization problems: Entropy-Bounded Information Maximization (EBIM) for the encoder, and Minimum Entropy Coupling (MEC) for the decoder. Through extensive analysis, we provide a greedy algorithm for EBIM with guaranteed performance, and characterize the optimal solution near functional mappings, yielding significant theoretical insights into the structural complexity of this problem. Furthermore, we illustrate the practical application of MEC-B through experiments in Markov Coding Games (MCGs) under rate limits. These games simulate a communication scenario within a Markov Decision Process, where an agent must transmit a compressed message from a sender to a receiver through its actions. Our experiments highlight the trade-offs between MDP rewards and receiver accuracy across various compression rates, showcasing the efficacy of our method compared to conventional compression baseline.