mlsanityData to 7 Oct 2026

On the advantages of stochastic encoders

arXiv 2102.09270 ↗First posted 18 Feb 2021cs.IT math.IT stat.MLcited by 28

Not on the map. Why?

Abstract (arXiv)

Stochastic encoders have been used in rate-distortion theory and neural compression because they can be easier to handle. However, in performance comparisons with deterministic encoders they often do worse, suggesting that noise in the encoding process may generally be a bad idea. It is poorly understood if and when stochastic encoders do better than deterministic encoders. In this paper we provide one illustrative example which shows that stochastic encoders can significantly outperform the best deterministic encoders. Our toy example suggests that stochastic encoders may be particularly useful in the regime of "perfect perceptual quality".

Key facts. Cited by 28 papers in the map (data through 7 Oct 2026). Builds on 7 of its 9 references that are in the map. First posted on arXiv 18 Feb 2021.

Builds on

7 of 9 references.

References · 7

  1. 2017The Perception-Distortion Tradeoffcited by 582
  2. 2019Rethinking Lossy Compression: The Rate-Distortion-Perception Tradeoffcited by 207
  3. 2020Nonlinear Transform Codingcited by 132
  4. 2020Universally Quantized Neural Compressioncited by 60
  5. 2018Deep Generative Models for Distribution-Preserving Lossy Compressioncited by 59
  6. 2018Minimal Random Code Learning: Getting Bits Back from Compressed Model Parameterscited by 56
  7. 2020Neural Networks Optimally Compress the Sawbridgecited by 11

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 3; 2022: 6; 2023: 5; 2024: 6; 2025: 5; 2026: 3.

28 papers.

Sort:

By area:

  1. 2022Beyond Transmitting Bits: Context, Semantics, and Task-Oriented Communicationscited by 166
  2. 2022Lossy Compression with Gaussian Diffusioncited by 87
  3. 2022Multi-Realism Image Compression with a Conditional Generatorcited by 79
  4. 2022An Introduction to Neural Data Compressioncited by 78
  5. 2023High-Fidelity Image Compression with Score-based Generative Modelscited by 53
  6. 2021Universal Rate-Distortion-Perception Representations for Lossy Compressioncited by 46
  7. 2021Neural Video Compression using GANs for Detail Synthesis and Propagationcited by 35
  8. 2021Algorithms for the Communication of Samplescited by 31
  9. 2023Compression with Bayesian Implicit Neural Representationscited by 25
  10. 2022The Rate-Distortion-Perception Tradeoff: The Role of Common Randomnesscited by 21
  11. 2023Just One Byte (per gradient): A Note on Low-Bandwidth Decentralized Language Model Finetuning Using Shared Randomnesscited by 18
  12. 2024What makes an image realistic?cited by 10
16 more
  1. 2024Rate-Distortion-Perception Tradeoff Based on the Conditional-Distribution Perception Measurecited by 8
  2. 2023Conditional Perceptual Quality Preserving Image Compressioncited by 6
  3. 2022Multi-Sample Training for Neural Image Compressioncited by 5
  4. 2024Output-Constrained Lossy Source Coding With Application to Rate-Distortion-Perception Theorycited by 5
  5. 2024Some Notes on the Sample Complexity of Approximate Channel Simulationcited by 4
  6. 2024Accelerating Relative Entropy Coding with Space Partitioningcited by 4
  7. 2024Gaussian Rate-Distortion-Perception Coding and Entropy-Constrained Scalar Quantizationcited by 4
  8. 2025Synonymous Variational Inference for Perceptual Image Compressioncited by 4
  9. 2023Rate-Distortion-Perception Theory for Semantic Communicationcited by 2
  10. 2025Optimal Neural Compressors for the Rate-Distortion-Perception Tradeoffcited by 2
  11. 2025Rate-Distortion-Perception Theory for the Quadratic Wasserstein Spacecited by 2
  12. 2026Deep Randomized Distributed Function Computation (DeepRDFC): Neural Distributed Channel Simulationcited by 1
  13. 2025Constrained Gaussian Wasserstein Optimal Transport with Commutative Covariance Matricescited by 0
  14. 2025Universal Representations for Classification-enhanced Lossy Compressioncited by 0
  15. 2026Advances in Diffusion-Based Generative Compressioncited by 0
  16. 2026Cross-Domain Lossy Compression via Constrained Minimum Entropy Couplingcited by 0