# CMC-Bench: Towards a New Paradigm of Visual Signal Compression

arXiv: https://arxiv.org/abs/2406.09356 · HTML: https://mlsanity.com/paper/2406.09356/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- Builds on 46 of its 76 references that are in the map.
- First posted on arXiv 13 Jun 2024.

## Builds on (most cited first, 30 of 46)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [High-Resolution Image Synthesis with Latent Diffusion Models](https://mlsanity.com/paper/2112.10752/) (2021)
- [Microsoft COCO: Common Objects in Context](https://mlsanity.com/paper/1405.0312/) (2014)
- [The Unreasonable Effectiveness of Deep Features as a Perceptual Metric](https://mlsanity.com/paper/1801.03924/) (2018)
- [Hierarchical Text-Conditional Image Generation with CLIP Latents](https://mlsanity.com/paper/2204.06125/) (2022)
- [Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding](https://mlsanity.com/paper/2205.11487/) (2022)
- [Adding Conditional Control to Text-to-Image Diffusion Models](https://mlsanity.com/paper/2302.05543/) (2023)
- [InstructPix2Pix: Learning to Follow Image Editing Instructions](https://mlsanity.com/paper/2211.09800/) (2022)
- [AnimateDiff: Animate Your Personalized Text-to-Image Diffusion Models without Specific Tuning](https://mlsanity.com/paper/2307.04725/) (2023)
- [ImageReward: Learning and Evaluating Human Preferences for Text-to-Image Generation](https://mlsanity.com/paper/2304.05977/) (2023)
- [Exploring CLIP for Assessing the Look and Feel of Images](https://mlsanity.com/paper/2207.12396/) (2022)
- [SEED-Bench: Benchmarking Multimodal LLMs with Generative Comprehension](https://mlsanity.com/paper/2307.16125/) (2023)
- [PixArt-α: Fast Training of Diffusion Transformer for Photorealistic Text-to-Image Synthesis](https://mlsanity.com/paper/2310.00426/) (2023)
- [Image Quality Assessment: Unifying Structure and Texture Similarity](https://mlsanity.com/paper/2004.07728/) (2020)
- [The Perception-Distortion Tradeoff](https://mlsanity.com/paper/1711.06077/) (2017)
- [Exploiting Diffusion Prior for Real-World Image Super-Resolution](https://mlsanity.com/paper/2305.07015/) (2023)
- [Blind Image Quality Assessment Using A Deep Bilinear Convolutional Neural Network](https://mlsanity.com/paper/1907.02665/) (2019)
- [Blind Image Quality Assessment via Vision-Language Correspondence: A Multitask Learning Perspective](https://mlsanity.com/paper/2303.14968/) (2023)
- [DiffBIR: Towards Blind Image Restoration with Generative Diffusion Prior](https://mlsanity.com/paper/2308.15070/) (2023)
- [Pixel-Aware Stable Diffusion for Realistic Image Super-resolution and Personalized Stylization](https://mlsanity.com/paper/2308.14469/) (2023)
- [Q-Bench: A Benchmark for General-Purpose Foundation Models on Low-level Vision](https://mlsanity.com/paper/2309.14181/) (2023)
- [DALL-Eval: Probing the Reasoning Skills and Social Biases of Text-to-Image Generation Models](https://mlsanity.com/paper/2202.04053/) (2022)
- [TOPIQ: A Top-down Approach from Semantics to Distortions for Image Quality Assessment](https://mlsanity.com/paper/2308.03060/) (2023)
- [Rethinking Lossy Compression: The Rate-Distortion-Perception Tradeoff](https://mlsanity.com/paper/1901.07821/) (2019)
- [Lossy Image Compression with Conditional Diffusion Models](https://mlsanity.com/paper/2209.06950/) (2022)
- [PieAPP: Perceptual Image-Error Assessment through Pairwise Preference](https://mlsanity.com/paper/1806.02067/) (2018)
- [AGIQA-3K: An Open Database for AI-Generated Image Quality Assessment](https://mlsanity.com/paper/2306.04717/) (2023)
- [Nonlinear Transform Coding](https://mlsanity.com/paper/2007.03034/) (2020)
- [HRS-Bench: Holistic, Reliable and Scalable Benchmark for Text-to-Image Models](https://mlsanity.com/paper/2304.05390/) (2023)
- [Text-Guided Synthesis of Artistic Images with Retrieval-Augmented Diffusion Models](https://mlsanity.com/paper/2207.13038/) (2022)

## Built on by (most cited first, 6 of 6)

- [R-Bench: Are your Large Multimodal Model Robust to Real-world Corruptions?](https://mlsanity.com/paper/2410.05474/) (2024)
- [Quality Assessment in the Era of Large Models: A Survey](https://mlsanity.com/paper/2409.00031/) (2024)
- [Video Quality Assessment: A Comprehensive Survey](https://mlsanity.com/paper/2412.04508/) (2024)
- [Image Quality Assessment: From Human to Machine Preference](https://mlsanity.com/paper/2503.10078/) (2025)
- [SynthVLM: Towards High-Quality and Efficient Synthesis of Image-Caption Datasets for Vision-Language Models](https://mlsanity.com/paper/2407.20756/) (2024)
- [Embodied Image Quality Assessment for Robotic Intelligence](https://mlsanity.com/paper/2412.18774/) (2024)

## Abstract (arXiv)

Ultra-low bitrate image compression is a challenging and demanding topic. With the development of Large Multimodal Models (LMMs), a Cross Modality Compression (CMC) paradigm of Image-Text-Image has emerged. Compared with traditional codecs, this semantic-level compression can reduce image data size to 0.1% or even lower, which has strong potential applications. However, CMC has certain defects in consistency with the original image and perceptual quality. To address this problem, we introduce CMC-Bench, a benchmark of the cooperative performance of Image-to-Text (I2T) and Text-to-Image (T2I) models for image compression. This benchmark covers 18,000 and 40,000 images respectively to verify 6 mainstream I2T and 12 T2I models, including 160,000 subjective preference scores annotated by human experts. At ultra-low bitrates, this paper proves that the combination of some I2T and T2I models has surpassed the most advanced visual signal codecs; meanwhile, it highlights where LMMs can be further optimized toward the compression task. We encourage LMM developers to participate in this test to promote the evolution of visual signal codec protocols.