# Progressive Learning with Visual Prompt Tuning for Variable-Rate Image Compression

arXiv: https://arxiv.org/abs/2311.13846 · HTML: https://mlsanity.com/paper/2311.13846/

## Key facts

- Cited by 3 papers in the map (data through 14 Oct 2026).
- Builds on 25 of its 33 references that are in the map.
- First posted on arXiv 23 Nov 2023.

## Builds on (most cited first, 25 of 25)

- [Language Models are Few-Shot Learners](https://mlsanity.com/paper/2005.14165/) (2020)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing](https://mlsanity.com/paper/2107.13586/) (2021)
- [Visual Prompt Tuning](https://mlsanity.com/paper/2203.12119/) (2022)
- [Variational image compression with a scale hyperprior](https://mlsanity.com/paper/1802.01436/) (2018)
- [End-to-end Optimized Image Compression](https://mlsanity.com/paper/1611.01704/) (2016)
- [Joint Autoregressive and Hierarchical Priors for Learned Image Compression](https://mlsanity.com/paper/1809.02736/) (2018)
- [Learned Image Compression with Discretized Gaussian Mixture Likelihoods and Attention Modules](https://mlsanity.com/paper/2001.01568/) (2020)
- [High-Fidelity Generative Image Compression](https://mlsanity.com/paper/2006.09965/) (2020)
- [Channel-wise Autoregressive Entropy Models for Learned Image Compression](https://mlsanity.com/paper/2007.08739/) (2020)
- [Full Resolution Image Compression with Recurrent Neural Networks](https://mlsanity.com/paper/1608.05148/) (2016)
- [Variable Rate Image Compression with Recurrent Neural Networks](https://mlsanity.com/paper/1511.06085/) (2015)
- [The Devil Is in the Details: Window-based Attention for Image Compression](https://mlsanity.com/paper/2203.08450/) (2022)
- [Context-adaptive Entropy Model for End-to-end Optimized Image Compression](https://mlsanity.com/paper/1809.10452/) (2018)
- [Variable Rate Deep Image Compression With a Conditional Autoencoder](https://mlsanity.com/paper/1909.04802/) (2019)
- [Entroformer: A Transformer-based Entropy Model for Learned Image Compression](https://mlsanity.com/paper/2202.05492/) (2022)
- [Enhanced Invertible Encoding for Learned Image Compression](https://mlsanity.com/paper/2108.03690/) (2021)
- [Asymmetric Gained Deep Image Compression With Continuous Rate Adaptation](https://mlsanity.com/paper/2003.02012/) (2020)
- [Transformer-based Image Compression](https://mlsanity.com/paper/2111.06707/) (2021)
- [Variable-Rate Deep Image Compression through Spatially-Adaptive Feature Transform](https://mlsanity.com/paper/2108.09551/) (2021)
- [Joint Global and Local Hierarchical Priors for Learned Image Compression](https://mlsanity.com/paper/2112.04487/) (2021)
- [Variable Rate Deep Image Compression with Modulated Autoencoder](https://mlsanity.com/paper/1912.05526/) (2019)
- [TransTIC: Transferring Transformer-based Image Compression from Human Perception to Machine Perception](https://mlsanity.com/paper/2306.05085/) (2023)
- [Flexible Neural Image Compression via Code Editing](https://mlsanity.com/paper/2209.09244/) (2022)
- [Instance-aware Prompt Learning for Language Understanding and Generation](https://mlsanity.com/paper/2201.07126/) (2022)

## Built on by (most cited first, 3 of 3)

- [Prompt-based Adaptation in Large-scale Vision Models: A Survey](https://mlsanity.com/paper/2510.13219/) (2025)
- [High Quality Underwater Image Compression with Adaptive Color Correction](https://mlsanity.com/paper/2505.09986/) (2025)
- [Variable-Rate Deep Image Compression based on Low-Rank Adaptation by Progressive Learning](https://mlsanity.com/paper/2606.16107/) (2026)

## Abstract (arXiv)

In this paper, we propose a progressive learning paradigm for transformer-based variable-rate image compression. Our approach covers a wide range of compression rates with the assistance of the Layer-adaptive Prompt Module (LPM). Inspired by visual prompt tuning, we use LPM to extract prompts for input images and hidden features at the encoder side and decoder side, respectively, which are fed as additional information into the Swin Transformer layer of a pre-trained transformer-based image compression model to affect the allocation of attention region and the bits, which in turn changes the target compression ratio of the model. To ensure the network is more lightweight, we involves the integration of prompt networks with less convolutional layers. Exhaustive experiments show that compared to methods based on multiple models, which are optimized separately for different target rates, the proposed method arrives at the same performance with 80% savings in parameter storage and 90% savings in datasets. Meanwhile, our model outperforms all current variable bitrate image methods in terms of rate-distortion performance and approaches the state-of-the-art fixed bitrate image compression methods trained from scratch.