# Ultra Lowrate Image Compression with Semantic Residual Coding and Compression-aware Diffusion

arXiv: https://arxiv.org/abs/2505.08281 · HTML: https://mlsanity.com/paper/2505.08281/

## Key facts

- Cited by 20 papers in the map (data through 14 Oct 2026).
- Placed in Diffusion for compression and wireless, one of 14 directions in Diffusion models.
- Builds on 45 of its 60 references that are in the map.
- First posted on arXiv 13 May 2025.

## Placed in

- Diffusion models / [Diffusion for compression and wireless](https://mlsanity.com/diffusion/diffusion-for-compression-and-wireless/)

## Builds on (most cited first, 30 of 45)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Denoising Diffusion Probabilistic Models](https://mlsanity.com/paper/2006.11239/) (2020)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [The Llama 3 Herd of Models](https://mlsanity.com/paper/2407.21783/) (2024)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Denoising Diffusion Implicit Models](https://mlsanity.com/paper/2010.02502/) (2020)
- [Visual Instruction Tuning](https://mlsanity.com/paper/2304.08485/) (2023)
- [Deep Unsupervised Learning using Nonequilibrium Thermodynamics](https://mlsanity.com/paper/1503.03585/) (2015)
- [Adding Conditional Control to Text-to-Image Diffusion Models](https://mlsanity.com/paper/2302.05543/) (2023)
- [BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation](https://mlsanity.com/paper/2201.12086/) (2022)
- [Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution](https://mlsanity.com/paper/2409.12191/) (2024)
- [DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation](https://mlsanity.com/paper/2208.12242/) (2022)
- [An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversion](https://mlsanity.com/paper/2208.01618/) (2022)
- [T2I-Adapter: Learning Adapters to Dig out More Controllable Ability for Text-to-Image Diffusion Models](https://mlsanity.com/paper/2302.08453/) (2023)
- [Demystifying MMD GANs](https://mlsanity.com/paper/1801.01401/) (2018)
- [Variational image compression with a scale hyperprior](https://mlsanity.com/paper/1802.01436/) (2018)
- [COCO-Stuff: Thing and Stuff Classes in Context](https://mlsanity.com/paper/1612.03716/) (2016)
- [Image Quality Assessment: Unifying Structure and Texture Similarity](https://mlsanity.com/paper/2004.07728/) (2020)
- [End-to-end Optimized Image Compression](https://mlsanity.com/paper/1611.01704/) (2016)
- [Joint Autoregressive and Hierarchical Priors for Learned Image Compression](https://mlsanity.com/paper/1809.02736/) (2018)
- [Exploiting Diffusion Prior for Real-World Image Super-Resolution](https://mlsanity.com/paper/2305.07015/) (2023)
- [ResShift: Efficient Diffusion Model for Image Super-resolution by Residual Shifting](https://mlsanity.com/paper/2307.12348/) (2023)
- [High-Fidelity Generative Image Compression](https://mlsanity.com/paper/2006.09965/) (2020)
- [Hard Prompts Made Easy: Gradient-Based Discrete Optimization for Prompt Tuning and Discovery](https://mlsanity.com/paper/2302.03668/) (2023)
- [ELIC: Efficient Learned Image Compression with Unevenly Grouped Space-Channel Contextual Adaptive Coding](https://mlsanity.com/paper/2203.10886/) (2022)
- [Pixel-Aware Stable Diffusion for Realistic Image Super-resolution and Personalized Stylization](https://mlsanity.com/paper/2308.14469/) (2023)
- [Lossy Image Compression with Conditional Diffusion Models](https://mlsanity.com/paper/2209.06950/) (2022)
- [Towards image compression with perfect realism at ultra-low bitrates](https://mlsanity.com/paper/2310.10325/) (2023)
- [Improving Statistical Fidelity for Neural Image Compression with Implicit Local Likelihood Models](https://mlsanity.com/paper/2301.11189/) (2023)
- [Non-local Attention Optimized Deep Image Compression](https://mlsanity.com/paper/1904.09757/) (2019)

## Built on by (most cited first, 20 of 20)

- [Generative Neural Video Compression via Video Diffusion Prior](https://mlsanity.com/paper/2512.05016/) (2025)
- [Ultra-Low Bitrate Perceptual Image Compression with Shallow Encoder](https://mlsanity.com/paper/2512.12229/) (2025)
- [CoD-Lite: Real-Time Diffusion-Based Generative Image Compression](https://mlsanity.com/paper/2604.12525/) (2026)
- [Dual-Representation Image Compression at Ultra-Low Bitrates via Explicit Semantics and Implicit Textures](https://mlsanity.com/paper/2602.05213/) (2026)
- [DiT-IC: Aligned Diffusion Transformer for Efficient Image Compression](https://mlsanity.com/paper/2603.13162/) (2026)
- [Differentiable Vector Quantization for Rate-Distortion Optimization of Generative Image Compression](https://mlsanity.com/paper/2604.10546/) (2026)
- [SLIM: Semantic-based Low-bitrate Image compression for Machines by leveraging diffusion](https://mlsanity.com/paper/2512.18200/) (2025)
- [Next-Frame Decoding for Ultra-Low-Bitrate Image Compression with Video Diffusion Priors](https://mlsanity.com/paper/2603.15129/) (2026)
- [Mage-Flow: An Efficient Native-Resolution Foundation Model for Image Generation and Editing](https://mlsanity.com/paper/2607.19064/) (2026)
- [CADC: Content Adaptive Diffusion-Based Generative Image Compression](https://mlsanity.com/paper/2602.21591/) (2026)
- [Towards Efficient Low-rate Image Compression with Frequency-aware Diffusion Prior Refinement](https://mlsanity.com/paper/2601.10373/) (2026)
- [Exploiting Semantic and Pixel Representations for Ultra-Low Bitrate Image Compression](https://mlsanity.com/paper/2606.01608/) (2026)
- [Training-Free Rate-Distortion-Perception Traversal With Diffusion](https://mlsanity.com/paper/2603.04005/) (2026)
- [Distributed Image Compression with Multimodal Side Information at Extremely Low Bitrates](https://mlsanity.com/paper/2605.22061/) (2026)
- [FlowCodec: One-Step Flow Prior for Generative Image Compression](https://mlsanity.com/paper/2606.21030/) (2026)
- [Rate-Adaptive One-Step Diffusion Compression for AIGC Images](https://mlsanity.com/paper/2609.31795/) (2026)
- [Rethinking Generative Image Compression at Extremely Low Bitrates](https://mlsanity.com/paper/2609.39315/) (2026)
- [ResARC: Residual-Aware AutoRegressive Coding for Ultra-Low Bitrate Image Compression](https://mlsanity.com/paper/2609.39451/) (2026)
- [Event-guided Neural Video Compression](https://mlsanity.com/paper/2610.02265/) (2026)
- [Recasting DiT for Extreme Image Compression with Hierarchical Visual Priors](https://mlsanity.com/paper/2610.18607/) (2026)

## Abstract (arXiv)

Existing multimodal large model-based image compression frameworks often rely on a fragmented integration of semantic retrieval, latent compression, and generative models, resulting in suboptimal performance in both reconstruction fidelity and coding efficiency. To address these challenges, we propose a residual-guided ultra lowrate image compression named ResULIC, which incorporates residual signals into both semantic retrieval and the diffusion-based generation process. Specifically, we introduce Semantic Residual Coding (SRC) to capture the semantic disparity between the original image and its compressed latent representation. A perceptual fidelity optimizer is further applied for superior reconstruction quality. Additionally, we present the Compression-aware Diffusion Model (CDM), which establishes an optimal alignment between bitrates and diffusion time steps, improving compression-reconstruction synergy. Extensive experiments demonstrate the effectiveness of ResULIC, achieving superior objective and subjective performance compared to state-of-the-art diffusion-based methods with - 80.7%, -66.3% BD-rate saving in terms of LPIPS and FID. Project page is available at https: //njuvision.github.io/ResULIC/.