mlsanityData to 7 Oct 2026

Lossy Image Compression with Foundation Diffusion Models

arXiv 2404.08580 ↗First posted 12 Apr 2024eess.IV cs.CVcited by 34

On the map: Diffusion models → Diffusion for compression and wireless

Abstract (arXiv)

Incorporating diffusion models in the image compression domain has the potential to produce realistic and detailed reconstructions, especially at extremely low bitrates. Previous methods focus on using diffusion models as expressive decoders robust to quantization errors in the conditioning signals, yet achieving competitive results in this manner requires costly training of the diffusion model and long inference times due to the iterative generative process. In this work we formulate the removal of quantization error as a denoising task, using diffusion to recover lost information in the transmitted image latent. Our approach allows us to perform less than 10% of the full diffusion generative process and requires no architectural changes to the diffusion model, enabling the use of foundation models as a strong prior without additional fine tuning of the backbone. Our proposed codec outperforms previous methods in quantitative realism metrics, and we verify that our reconstructions are qualitatively preferred by end users, even when other methods use twice the bitrate.

Key facts. Cited by 34 papers in the map (data through 7 Oct 2026). Placed in Diffusion for compression and wireless, one of 14 directions in Diffusion models. Builds on 28 of its 44 references that are in the map. First posted on arXiv 12 Apr 2024.

Builds on

28 of 44 references.

Closest ancestors · 2

Same direction, newest first

  1. 2023High-Fidelity Image Compression with Score-based Generative Models26 May
  2. 2022Lossy Image Compression with Conditional Diffusion Models14 Sep

Other references · 26

  1. 2020Denoising Diffusion Probabilistic Modelscited by 20,851
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  3. 2020Denoising Diffusion Implicit Modelscited by 9,655
  4. 2021Diffusion Models Beat GANs on Image Synthesiscited by 8,185
  5. 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
  6. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
  7. 2021On the Opportunities and Risks of Foundation Modelscited by 3,961
  8. 2021SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equationscited by 1,903
  9. 2023Align your Latents: High-Resolution Video Synthesis with Latent Diffusion Modelscited by 1,465
  10. 2018Variational image compression with a scale hyperpriorcited by 910
  11. 2016End-to-end Optimized Image Compressioncited by 673
  12. 2018Joint Autoregressive and Hierarchical Priors for Learned Image Compressioncited by 642
14 more
  1. 2017Video Enhancement with Task-Oriented Flowcited by 612
  2. 2023Simple diffusion: End-to-end diffusion for high resolution imagescited by 413
  3. 2020High-Fidelity Generative Image Compressioncited by 331
  4. 2022ELIC: Efficient Learned Image Compression with Unevenly Grouped Space-Channel Contextual Adaptive Codingcited by 296
  5. 2022Diffusion models as plug-and-play priorscited by 230
  6. 2018Generative Adversarial Networks for Extreme Learned Image Compressioncited by 213
  7. 2019Rethinking Lossy Compression: The Rate-Distortion-Perception Tradeoffcited by 207
  8. 2023Exposing flaws of generative model evaluation metrics and their unfair treatment of diffusion modelscited by 199
  9. 2022Scalable Adaptive Computation for Iterative Generationcited by 190
  10. 2022Entroformer: A Transformer-based Entropy Model for Learned Image Compressioncited by 102
  11. 2023Towards image compression with perfect realism at ultra-low bitratescited by 99
  12. 2023Improving Statistical Fidelity for Neural Image Compression with Implicit Local Likelihood Modelscited by 90
  13. 2022Multi-Realism Image Compression with a Conditional Generatorcited by 79
  14. 2023Text + Sketch: Image Compression at Ultra Low Ratescited by 63

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2024: 6; 2025: 15; 2026: 13.

34 papers.

Sort:

By area:

  1. 2025StableCodec: Taming One-Step Diffusion for Extreme Image Compressioncited by 29
  2. 2024BetterDepth: Plug-and-Play Diffusion Refiner for Zero-Shot Monocular Depth Estimationcited by 28
  3. 2025A Survey of Multimodal Hallucination Evaluation and Detectioncited by 26
  4. 2025One-Step Diffusion-Based Image Compression with Semantic Distillationcited by 23
  5. 2024Lossless data compression by large modelscited by 20
  6. 2024PerCo (SD): Open Perceptual Compressioncited by 16
  7. 2025PICD: Versatile Perceptual Image Compression with Diffusion Renderingcited by 10
  8. 2025GIViC: Generative Implicit Video Compressioncited by 9
  9. 2025Bridging the Gap between Gaussian Diffusion Models and Universal Quantization for Image Compressioncited by 7
  10. 2024PSC: Posterior Sampling-Based Compressioncited by 5
  11. 2024Timestep-Aware Diffusion Model for Extreme Image Rescalingcited by 5
  12. 2024High Frequency Matters: Uncertainty Guided Image Compression with Wavelet Diffusioncited by 4
22 more
  1. 2025HDCompression: Hybrid-Diffusion Image Compression for Ultra-Low Bitratescited by 4
  2. 2025Diff-ICMH: Harmonizing Machine and Human Vision in Image Compression with Generative Priorcited by 4
  3. 2025Ultra-Low Bitrate Perceptual Image Compression with Shallow Encodercited by 4
  4. 2025Guaranteed Conditional Diffusion: 3D Block-based Models for Scientific Data Compressioncited by 3
  5. 2025Steering One-Step Diffusion Model with Fidelity-Rich Decoder for Fast Image Compressioncited by 3
  6. 2026DiT-IC: Aligned Diffusion Transformer for Efficient Image Compressioncited by 3
  7. 2025Single-step Diffusion for Image Compression at Ultra-Low Bitratescited by 2
  8. 2025Generative Latent Diffusion for Efficient Spatiotemporal Data Reductioncited by 2
  9. 2026Next-Frame Decoding for Ultra-Low-Bitrate Image Compression with Video Diffusion Priorscited by 2
  10. 2026GVCC: Zero-Shot Video Compression via Codebook-Driven Stochastic Rectified Flowcited by 2
  11. 2026CADC: Content Adaptive Diffusion-Based Generative Image Compressioncited by 1
  12. 2026TextBoost: Boosting Scene Text Fidelity in Ultra-low Bitrate Image Compressioncited by 1
  13. 2025FaSDiff: Balancing Perception and Semantics in Face Compression via Stable Diffusion Priorscited by 0
  14. 2025Keep It Real: Challenges in Attacking Compression-Based Adversarial Purificationcited by 0
  15. 2026Advances in Diffusion-Based Generative Compressioncited by 0
  16. 2026Region-Adaptive Generative Compression with Spatially Varying Diffusion Modelscited by 0
  17. 2026DiV-INR: Extreme Low-Bitrate Diffusion Video Compression with INR Conditioningcited by 0
  18. 2026Active Sampling for Ultra-Low-Bit-Rate Video Compression via Conditional Controlled Diffusioncited by 0
  19. 2026Adaptive Fused Prior Transfer for Controllable Generative Image Compressioncited by 0
  20. 2026Group-of-Latents: Perceptual Video Compression at Extreme Bitrates via Masked Latent Generative Modelingcited by 0
  21. 2026FLM: Frequency-Aware Language Models for Generative Image Compressioncited by 0
  22. 2026ResARC: Residual-Aware AutoRegressive Coding for Ultra-Low Bitrate Image Compressioncited by 0

Similar, not cited

Close in wording, no citation either way

  1. 2025OSCAR: One-Step Diffusion Codec Across Multiple Bit-rates
  2. 2024Progressive Compression with Universally Quantized Diffusion Models
  3. 2025Compressed Image Generation with Denoising Diffusion Codebook Models
  4. 2022Lossy Compression with Gaussian Diffusion
  5. 2025Lossy Compression with Pretrained Diffusion Models
  6. 2024Diffusion Product Quantization
  7. 2025CoD: A Diffusion Foundation Model for Image Compression
  8. 2024Towards Extreme Image Compression with Latent Feature Guidance and Diffusion Prior