PerCoV2: Ultra-Low Bit-Rate Perceptual Image Compression via Query-Based 1D Multimodal Image Tokens
Not on the map. Why?
Abstract (arXiv)
Despite recent progress in learned image compression, current image codecs still struggle to maintain realistic reconstructions at low bit-rates, often producing structured artifacts such as grid patterns or repetitive textures, even when trained with perceptual or adversarial losses. We introduce PerCoV2, an ultra-low bit-rate perceptual image compression system that unifies semantic tokenization, flow-based generation, and learned entropy modeling within a single framework. Building on the fully open flow-based SANA architecture, PerCoV2 introduces a novel resolution-adaptive 1D query-based tokenizer that produces compact semantic image tokens with a dual role in flow matching: providing a data-dependent reconstruction prior for initialization and a conditioning signal for flow-based refinement. By explicitly decoupling semantic representation from perceptual generation, our dual representation simplifies the flow-based learning objective, leading to more stable optimization and improved perceptual compression performance. PerCoV2 further introduces a dedicated 1D masked entropy model to improve rate efficiency and optional decoder-side multimodal enhancement via a vision-language model (Molmo) without increasing the transmitted bit budget. On MSCOCO-30k, PerCoV2 achieves state-of-the-art statistical fidelity, measured by FID and KID, across ultra-low and extreme bit-rates (0.0015-0.025 bpp). When trained solely on the general-purpose SA-1B dataset, PerCoV2 further demonstrates strong zero-shot generalization to widely adopted high-resolution benchmarks, including DIV2K and CLIC 2020, achieving competitive statistical fidelity with the current leading method, AEIC-ME. Finally, we introduce PerCoV2-distilled, a practical single-step variant derived from multi-step flow matching that accelerates decoding by 5.37x over PerCoV1, while preserving perceptual compression performance.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Builds on 63 of its 103 references that are in the map. First posted on arXiv 12 Mar 2025.
Builds on
63 of 103 references.
References · 63
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,456
- 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
- 2020Denoising Diffusion Probabilistic Modelscited by 20,851
- 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,491
- 2021LoRA: Low-Rank Adaptation of Large Language Modelscited by 14,084
- 2018The Unreasonable Effectiveness of Deep Features as a Perceptual Metriccited by 11,029
- 2014ImageNet Large Scale Visual Recognition Challengecited by 8,441
- 2015Deep Unsupervised Learning using Nonequilibrium Thermodynamicscited by 7,005
- 2022Classifier-Free Diffusion Guidancecited by 5,717
- 2017Neural Discrete Representation Learningcited by 4,650
- 2024Scaling Rectified Flow Transformers for High-Resolution Image Synthesiscited by 4,094
- 2021On the Opportunities and Risks of Foundation Modelscited by 3,961
38 more
- 2021GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Modelscited by 3,595
- 2022Flow Straight and Fast: Learning to Generate and Transfer Data with Rectified Flowcited by 3,550
- 2020Taming Transformers for High-Resolution Image Synthesiscited by 3,229
- 2021CLIPScore: A Reference-free Evaluation Metric for Image Captioningcited by 2,388
- 2022An Image is Worth One Word: Personalizing Text-to-Image Generation using Textual Inversioncited by 2,231
- 2024Gemma 2: Improving Open Language Models at a Practical Sizecited by 1,928
- 2018Demystifying MMD GANscited by 1,268
- 2021Cascaded Diffusion Models for High Fidelity Image Generationcited by 1,155
- 2024Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale Predictioncited by 1,005
- 2023One-step Diffusion with Distribution Matching Distillationcited by 978
- 2022Building Normalizing Flows with Stochastic Interpolantscited by 964
- 2018Variational image compression with a scale hyperpriorcited by 910
- 2016COCO-Stuff: Thing and Stuff Classes in Contextcited by 858
- 2016End-to-end Optimized Image Compressioncited by 673
- 2024Representation Alignment for Generation: Training Diffusion Transformers Is Easier Than You Thinkcited by 666
- 2024Improved Distribution Matching Distillation for Fast Image Synthesiscited by 648
- 2022Vision Transformer Adapter for Dense Predictionscited by 564
- 2017Understanding the Effective Receptive Field in Deep Convolutional Neural Networkscited by 549
- 2023Gated Linear Attention Transformers with Hardware-Efficient Trainingcited by 490
- 2020High-Fidelity Generative Image Compressioncited by 331
- 2023Hard Prompts Made Easy: Gradient-Based Discrete Optimization for Prompt Tuning and Discoverycited by 329
- 2024Infinity: Scaling Bitwise AutoRegressive Modeling for High-Resolution Image Synthesiscited by 297
- 2022ELIC: Efficient Learned Image Compression with Unevenly Grouped Space-Channel Contextual Adaptive Codingcited by 296
- 2024An Image is Worth 32 Tokens for Reconstruction and Generationcited by 295
- 2017Lossy Image Compression with Compressive Autoencoderscited by 289
- 2020Channel-wise Autoregressive Entropy Models for Learned Image Compressioncited by 255
- 2023DiffBIR: Towards Blind Image Restoration with Generative Diffusion Priorcited by 255
- 2018Generative Adversarial Networks for Extreme Learned Image Compressioncited by 213
- 2019Rethinking Lossy Compression: The Rate-Distortion-Perception Tradeoffcited by 207
- 2020You Only Need Adversarial Supervision for Semantic Image Synthesiscited by 129
- 2023Towards image compression with perfect realism at ultra-low bitratescited by 99
- 2023Stochastic interpolants with data-dependent couplingscited by 94
- 2023Improving Statistical Fidelity for Neural Image Compression with Implicit Local Likelihood Modelscited by 90
- 2022An Introduction to Neural Data Compressioncited by 78
- 2025Towards Practical Real-Time Neural Video Compressioncited by 72
- 2023Text + Sketch: Image Compression at Ultra Low Ratescited by 63
- 2018Deep Generative Models for Distribution-Preserving Lossy Compressioncited by 59
- 2024Towards Extreme Image Compression with Latent Feature Guidance and Diffusion Priorcited by 56
50 most cited of 63 All 63 (JSON)
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2024PerCo (SD): Open Perceptual Compression
- 2025Single-step Diffusion for Image Compression at Ultra-Low Bitrates
- 2026One-Step Diffusion for Perceptual Image Compression
- 2026Exploiting Semantic and Pixel Representations for Ultra-Low Bitrate Image Compression
- 2026Region-Adaptive Generative Compression with Spatially Varying Diffusion Models
- 2025Ultra Lowrate Image Compression with Semantic Residual Coding and Compression-aware Diffusion
- 2026Dual-Representation Image Compression at Ultra-Low Bitrates via Explicit Semantics and Implicit Textures
- 2025HDCompression: Hybrid-Diffusion Image Compression for Ultra-Low Bitrates