# On the advantages of stochastic encoders

arXiv: https://arxiv.org/abs/2102.09270 · HTML: https://mlsanity.com/paper/2102.09270/

## Key facts

- Cited by 28 papers in the map (data through 14 Oct 2026).
- Builds on 7 of its 9 references that are in the map.
- First posted on arXiv 18 Feb 2021.

## Builds on (most cited first, 7 of 7)

- [The Perception-Distortion Tradeoff](https://mlsanity.com/paper/1711.06077/) (2017)
- [Rethinking Lossy Compression: The Rate-Distortion-Perception Tradeoff](https://mlsanity.com/paper/1901.07821/) (2019)
- [Nonlinear Transform Coding](https://mlsanity.com/paper/2007.03034/) (2020)
- [Universally Quantized Neural Compression](https://mlsanity.com/paper/2006.09952/) (2020)
- [Deep Generative Models for Distribution-Preserving Lossy Compression](https://mlsanity.com/paper/1805.11057/) (2018)
- [Minimal Random Code Learning: Getting Bits Back from Compressed Model Parameters](https://mlsanity.com/paper/1810.00440/) (2018)
- [Neural Networks Optimally Compress the Sawbridge](https://mlsanity.com/paper/2011.05065/) (2020)

## Built on by (most cited first, 28 of 28)

- [Beyond Transmitting Bits: Context, Semantics, and Task-Oriented Communications](https://mlsanity.com/paper/2207.09353/) (2022)
- [Lossy Compression with Gaussian Diffusion](https://mlsanity.com/paper/2206.08889/) (2022)
- [Multi-Realism Image Compression with a Conditional Generator](https://mlsanity.com/paper/2212.13824/) (2022)
- [An Introduction to Neural Data Compression](https://mlsanity.com/paper/2202.06533/) (2022)
- [High-Fidelity Image Compression with Score-based Generative Models](https://mlsanity.com/paper/2305.18231/) (2023)
- [Universal Rate-Distortion-Perception Representations for Lossy Compression](https://mlsanity.com/paper/2106.10311/) (2021)
- [Neural Video Compression using GANs for Detail Synthesis and Propagation](https://mlsanity.com/paper/2107.12038/) (2021)
- [Algorithms for the Communication of Samples](https://mlsanity.com/paper/2110.12805/) (2021)
- [Compression with Bayesian Implicit Neural Representations](https://mlsanity.com/paper/2305.19185/) (2023)
- [The Rate-Distortion-Perception Tradeoff: The Role of Common Randomness](https://mlsanity.com/paper/2202.04147/) (2022)
- [Just One Byte (per gradient): A Note on Low-Bandwidth Decentralized Language Model Finetuning Using Shared Randomness](https://mlsanity.com/paper/2306.10015/) (2023)
- [What makes an image realistic?](https://mlsanity.com/paper/2403.04493/) (2024)
- [Rate-Distortion-Perception Tradeoff Based on the Conditional-Distribution Perception Measure](https://mlsanity.com/paper/2401.12207/) (2024)
- [Conditional Perceptual Quality Preserving Image Compression](https://mlsanity.com/paper/2308.08154/) (2023)
- [Multi-Sample Training for Neural Image Compression](https://mlsanity.com/paper/2209.13834/) (2022)
- [Output-Constrained Lossy Source Coding With Application to Rate-Distortion-Perception Theory](https://mlsanity.com/paper/2403.14849/) (2024)
- [Some Notes on the Sample Complexity of Approximate Channel Simulation](https://mlsanity.com/paper/2405.04363/) (2024)
- [Accelerating Relative Entropy Coding with Space Partitioning](https://mlsanity.com/paper/2405.12203/) (2024)
- [Gaussian Rate-Distortion-Perception Coding and Entropy-Constrained Scalar Quantization](https://mlsanity.com/paper/2409.02388/) (2024)
- [Synonymous Variational Inference for Perceptual Image Compression](https://mlsanity.com/paper/2505.22438/) (2025)
- [Rate-Distortion-Perception Theory for Semantic Communication](https://mlsanity.com/paper/2312.05437/) (2023)
- [Optimal Neural Compressors for the Rate-Distortion-Perception Tradeoff](https://mlsanity.com/paper/2503.17558/) (2025)
- [Rate-Distortion-Perception Theory for the Quadratic Wasserstein Space](https://mlsanity.com/paper/2504.17236/) (2025)
- [Advances in Diffusion-Based Generative Compression](https://mlsanity.com/paper/2601.18932/) (2026)
- [Deep Randomized Distributed Function Computation (DeepRDFC): Neural Distributed Channel Simulation](https://mlsanity.com/paper/2603.10750/) (2026)
- [Constrained Gaussian Wasserstein Optimal Transport with Commutative Covariance Matrices](https://mlsanity.com/paper/2503.03744/) (2025)
- [Universal Representations for Classification-enhanced Lossy Compression](https://mlsanity.com/paper/2504.13191/) (2025)
- [Cross-Domain Lossy Compression via Constrained Minimum Entropy Coupling](https://mlsanity.com/paper/2605.09833/) (2026)

## Abstract (arXiv)

Stochastic encoders have been used in rate-distortion theory and neural compression because they can be easier to handle. However, in performance comparisons with deterministic encoders they often do worse, suggesting that noise in the encoding process may generally be a bad idea. It is poorly understood if and when stochastic encoders do better than deterministic encoders. In this paper we provide one illustrative example which shows that stochastic encoders can significantly outperform the best deterministic encoders. Our toy example suggests that stochastic encoders may be particularly useful in the regime of "perfect perceptual quality".