mlsanityData to 7 Oct 2026

Text + Sketch: Image Compression at Ultra Low Rates

arXiv 2307.01944 ↗First posted 4 Jul 2023cs.LG cs.CV cs.ITcited by 63

Not on the map. Why?

Cited by 31 in Diffusion models, but placed by its own abstract, which didn’t fit.

Abstract (arXiv)

Recent advances in text-to-image generative models provide the ability to generate high-quality images from short text descriptions. These foundation models, when pre-trained on billion-scale datasets, are effective for various downstream tasks with little or no further training. A natural question to ask is how such models may be adapted for image compression. We investigate several techniques in which the pre-trained models can be directly used to implement compression schemes targeting novel low rate regimes. We show how text descriptions can be used in conjunction with side information to generate high-fidelity reconstructions that preserve both semantics and spatial structure of the original. We demonstrate that at very low bit-rates, our method can significantly improve upon learned compressors in terms of perceptual and semantic fidelity, despite no end-to-end training.

Key facts. Cited by 63 papers in the map (data through 7 Oct 2026). Builds on 27 of its 42 references that are in the map. First posted on arXiv 4 Jul 2023.

Builds on

27 of 42 references.

References · 27

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
  2. 2021High-Resolution Image Synthesis with Latent Diffusion Modelscited by 18,694
  3. 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
  4. 2022Hierarchical Text-Conditional Image Generation with CLIP Latentscited by 6,539
  5. 2023Adding Conditional Control to Text-to-Image Diffusion Modelscited by 5,466
  6. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,310
  7. 2021Zero-Shot Text-to-Image Generationcited by 4,276
  8. 2018Demystifying MMD GANscited by 1,268
  9. 2021CogView: Mastering Text-to-Image Generation via Transformerscited by 691
  10. 2016End-to-end Optimized Image Compressioncited by 673
  11. 2017Video Enhancement with Task-Oriented Flowcited by 612
  12. 2015Holistically-Nested Edge Detectioncited by 590
15 more
  1. 2020Learned Image Compression with Discretized Gaussian Mixture Likelihoods and Attention Modulescited by 570
  2. 2020High-Fidelity Generative Image Compressioncited by 331
  3. 2023Hard Prompts Made Easy: Gradient-Based Discrete Optimization for Prompt Tuning and Discoverycited by 329
  4. 2017Lossy Image Compression with Compressive Autoencoderscited by 289
  5. 2018Generative Adversarial Networks for Extreme Learned Image Compressioncited by 213
  6. 2019Rethinking Lossy Compression: The Rate-Distortion-Perception Tradeoffcited by 207
  7. 2015Variable Rate Image Compression with Recurrent Neural Networkscited by 168
  8. 2022Lossy Image Compression with Conditional Diffusion Modelscited by 165
  9. 2021StyleGAN-NADA: CLIP-Guided Domain Adaptation of Image Generatorscited by 164
  10. 2017Soft-to-Hard Vector Quantization for End-to-End Learning Compressible Representationscited by 149
  11. 2021From Show to Tell: A Survey on Deep Learning-based Image Captioningcited by 137
  12. 2020Nonlinear Transform Codingcited by 132
  13. 2022Lossy Compression with Gaussian Diffusioncited by 87
  14. 2022Extreme Generative Image Compression by Learning Text Embedding from Diffusion Modelscited by 30
  15. 2018Towards improved lossy image compression: Human image reconstruction with public-domain imagescited by 3

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2023: 1; 2024: 16; 2025: 28; 2026: 18.

63 papers.

Sort:

By area:

  1. 2023Towards image compression with perfect realism at ultra-low bitratescited by 99
  2. 2024Towards Extreme Image Compression with Latent Feature Guidance and Diffusion Priorcited by 56
  3. 2024Lossy Image Compression with Foundation Diffusion Modelscited by 34
  4. 2024MISC: Ultra-low Bitrate Image Semantic Compression Driven by Large Multimodal Modelcited by 29
  5. 2025StableCodec: Taming One-Step Diffusion for Extreme Image Compressioncited by 29
  6. 2025Lossy Compression with Pretrained Diffusion Modelscited by 23
  7. 2025One-Step Diffusion-Based Image Compression with Semantic Distillationcited by 23
  8. 2025Generative Latent Coding for Ultra-Low Bitrate Image and Video Compressioncited by 22
  9. 2024RDEIC: Accelerating Diffusion-Based Extreme Image Compression with Relay Residual Diffusioncited by 21
  10. 2025Generative Latent Coding for Ultra-Low Bitrate Image Compressioncited by 21
  11. 2024Neural Image Compression with Text-guided Encoding for both Pixel-level and Perceptual Fidelitycited by 20
  12. 2025DLF: Extreme Image Compression with Dual-generative Latent Fusioncited by 20
38 more
  1. 2025Ultra Lowrate Image Compression with Semantic Residual Coding and Compression-aware Diffusioncited by 17
  2. 2024PerCo (SD): Open Perceptual Compressioncited by 16
  3. 2025Semantics-Guided Diffusion for Deep Joint Source-Channel Coding in Wireless Image Transmissioncited by 15
  4. 2024Diffusion-Aided Joint Source Channel Coding For High Realism Wireless Image Transmissioncited by 11
  5. 2025Generative Neural Video Compression via Video Diffusion Priorcited by 11
  6. 2025PICD: Versatile Perceptual Image Compression with Diffusion Renderingcited by 10
  7. 2025Generative Video Semantic Communication via Multimodal Semantic Fusion with Large Modelcited by 9
  8. 2025CoD: A Diffusion Foundation Model for Image Compressioncited by 9
  9. 2025Bridging the Gap between Gaussian Diffusion Models and Universal Quantization for Image Compressioncited by 7
  10. 2024Semantic Similarity Score for Measuring Visual Similarity at Semantic Levelcited by 5
  11. 2024Semantic Feature Decomposition based Semantic Communication System of Images with Large-scale Visual Generation Modelscited by 4
  12. 2024LMM-driven Semantic Image-Text Coding for Ultra Low-bitrate Learned Image Compressioncited by 4
  13. 2025Extremely low-bitrate Image Compression Semantically Disentangled by LMMs from a Human Perception Perspectivecited by 4
  14. 2025Ultra-Low Bitrate Perceptual Image Compression with Shallow Encodercited by 4
  15. 2026Diffusion-aided Extreme Video Compression with Lightweight Semantics Guidancecited by 4
  16. 2026Dual-Representation Image Compression at Ultra-Low Bitrates via Explicit Semantics and Implicit Texturescited by 4
  17. 2026Differentiable Vector Quantization for Rate-Distortion Optimization of Generative Image Compressioncited by 4
  18. 2026CoD-Lite: Real-Time Diffusion-Based Generative Image Compressioncited by 4
  19. 2024Map-Assisted Remote-Sensing Image Compression at Extremely Low Bitratescited by 3
  20. 2025Steering One-Step Diffusion Model with Fidelity-Rich Decoder for Fast Image Compressioncited by 3
  21. 2024Exploring the Limits of Semantic Image Compression at Micro-bits per Pixelcited by 2
  22. 2025Guided Diffusion for the Extension of Machine Vision to Human Visual Perceptioncited by 2
  23. 2025Why Compress What You Can Generate? When GPT-4o Generation Ushers in Image Compression Fieldscited by 2
  24. 2026Investigating the Development of Task-Oriented Communication in Vision-Language Modelscited by 2
  25. 2026Next-Frame Decoding for Ultra-Low-Bitrate Image Compression with Video Diffusion Priorscited by 2
  26. 2024Exploiting Inter-Image Similarity Prior for Low-Bitrate Remote Sensing Image Compressioncited by 1
  27. 2025Hierarchical Semantic Compression for Consistent Image Semantic Restorationcited by 1
  28. 2025Generative Image Compression by Estimating Gradients of the Rate-variable Feature Distributioncited by 1
  29. 2025Generative Image Coding with Diffusion Priorcited by 1
  30. 2025Switchable Token-Specific Codebook Quantization For Face Image Compressioncited by 1
  31. 2025Machines Serve Human: A Novel Variable Human-machine Collaborative Compression Frameworkcited by 1
  32. 2025VLIC: Vision-Language Models As Perceptual Judges for Human-Aligned Image Compressioncited by 1
  33. 2026TextBoost: Boosting Scene Text Fidelity in Ultra-low Bitrate Image Compressioncited by 1
  34. 2026Exploiting Semantic and Pixel Representations for Ultra-Low Bitrate Image Compressioncited by 1
  35. 2024Fine color guidance in diffusion models and its application to image compression at extremely low bitratescited by 0
  36. 2024SMIC: Semantic Multi-Item Compression based on CLIP dictionarycited by 0
  37. 2024Semantics Disentanglement and Composition for Universal Image Coding with Efficiently LLM Reasoning and Generative Diffusioncited by 0
  38. 2025PerCoV2: Ultra-Low Bit-Rate Perceptual Image Compression via Query-Based 1D Multimodal Image Tokenscited by 0

50 most cited of 63 All 63 (JSON)