mlsanityData to 7 Oct 2026

Neural Distributed Source Coding

arXiv 2106.02797 ↗First posted 5 Jun 2021cs.IT cs.LG math.ITcited by 13

Not on the map. Why?

Abstract (arXiv)

Distributed source coding (DSC) is the task of encoding an input in the absence of correlated side information that is only available to the decoder. Remarkably, Slepian and Wolf showed in 1973 that an encoder without access to the side information can asymptotically achieve the same compression rate as when the side information is available to it. While there is vast prior work on this topic, practical DSC has been limited to synthetic datasets and specific correlation structures. Here we present a framework for lossy DSC that is agnostic to the correlation structure and can scale to high dimensions. Rather than relying on hand-crafted source modeling, our method utilizes a conditional Vector-Quantized Variational Autoencoder (VQ-VAE) to learn the distributed encoder and decoder. We evaluate our method on multiple datasets and show that our method can handle complex correlations and achieves state-of-the-art PSNR. Our code is made available at https://github.com/acnagle/neural-dsc.

Key facts. Cited by 13 papers in the map (data through 7 Oct 2026). Builds on 35 of its 77 references that are in the map. First posted on arXiv 5 Jun 2021.

Builds on

35 of 77 references.

References · 35

  1. 2017Attention Is All You Needcited by 59,932
  2. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  3. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  4. 2017Neural Discrete Representation Learningcited by 4,650
  5. 2021Zero-Shot Text-to-Image Generationcited by 4,276
  6. 2014Deep Learning Face Attributes in the Wildcited by 3,679
  7. 2018Glow: Generative Flow with Invertible 1x1 Convolutionscited by 1,539
  8. 2019Generating Diverse High-Fidelity Images with VQ-VAE-2cited by 1,376
  9. 2019An Introduction to Variational Autoencoderscited by 997
  10. 2018Variational image compression with a scale hyperpriorcited by 910
  11. 2016Conditional Image Generation with PixelCNN Decoderscited by 859
  12. 2016End-to-end Optimized Image Compressioncited by 673
23 more
  1. 2018Joint Autoregressive and Hierarchical Priors for Learned Image Compressioncited by 642
  2. 2018signSGD: Compressed Optimisation for Non-Convex Problemscited by 561
  3. 2017Deep Gradient Compression: Reducing the Communication Bandwidth for Distributed Trainingcited by 471
  4. 2018Sparsified SGD with Memorycited by 373
  5. 2017Lossy Image Compression with Compressive Autoencoderscited by 289
  6. 2019Flow++: Improving Flow-Based Generative Models with Variational Dequantization and Architecture Designcited by 258
  7. 2019PowerSGD: Practical Low-Rank Gradient Compression for Distributed Optimizationcited by 234
  8. 2016Unrolled Generative Adversarial Networkscited by 208
  9. 2017VEEGAN: Reducing Mode Collapse in GANs using Implicit Variational Learningcited by 199
  10. 2019Qsparse-local-SGD: Distributed SGD with Quantization, Sparsification, and Local Computationscited by 179
  11. 2017PixelSNAIL: An Improved Autoregressive Generative Modelcited by 137
  12. 2018ATOMO: Communication-efficient Learning via Atomic Sparsificationcited by 135
  13. 2020Nonlinear Transform Codingcited by 132
  14. 2018A Hierarchical Latent Vector Model for Learning Long-Term Structure in Musiccited by 131
  15. 2019Decentralized Deep Learning with Arbitrary Communication Compressioncited by 109
  16. 2019Practical Lossless Compression with Latent Variables using Bits Back Codingcited by 85
  17. 2018Minimal Random Code Learning: Getting Bits Back from Compressed Model Parameterscited by 56
  18. 2019Understanding Top-k Sparsification in Distributed Deep Learningcited by 56
  19. 2019Low Bit-Rate Speech Coding with VQ-VAE and a WaveNet Decodercited by 47
  20. 2019Turbo Autoencoder: Deep learning based channel codes for point-to-point communication channelscited by 42
  21. 2019A Distributed Synchronous SGD Algorithm with Global Top-k Sparsification for Low Bandwidth Networkscited by 36
  22. 2020Deep Image Compression using Decoder Side Informationcited by 14
  23. 2020On the Benefits of Multiple Gossip Steps in Communication-Constrained Decentralized Optimizationcited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2021: 2; 2022: 2; 2023: 5; 2024: 1; 2025: 1; 2026: 2.

13 papers.

Sort:

By area:

  1. 2021Task-Oriented Communication for Multi-Device Cooperative Edge Inferencecited by 47
  2. 2022Neural Distributed Image Compression with Cross-Attention Feature Alignmentcited by 15
  3. 2023LDMIC: Learning-based Distributed Multi-view Image Codingcited by 13
  4. 2021Neural Distributed Image Compression using Common Informationcited by 11
  5. 2023Task-aware Distributed Source Coding under Dynamic Bandwidthcited by 6
  6. 2025List-Level Distribution Coupling with Applications to Speculative Decoding and Lossy Compressioncited by 3
  7. 2022Task-Aware Network Coding Over Butterfly Networkcited by 1
  8. 2023Federated Neural Compression Under Heterogeneous Datacited by 1
  9. 2026V2X-DSC: Multi-Agent Collaborative Perception with Distributed Source Coding Guided Communicationcited by 1
  10. 2023Interpreting Training Aspects of Deep-Learned Error-Correcting Codescited by 0
  11. 2023Learned layered coding for Successive Refinement in the Wyner-Ziv Problemcited by 0
  12. 2024Image Compression Using Novel View Synthesis Priorscited by 0
1 more
  1. 2026Distributed Image Compression with Multimodal Side Information at Extremely Low Bitratescited by 0