# Prompt-Guided Prefiltering for VLM Image Compression

arXiv: https://arxiv.org/abs/2604.00314 · HTML: https://mlsanity.com/paper/2604.00314/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Placed in Visual token pruning and efficiency, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 24 of its 32 references that are in the map.
- First posted on arXiv 31 Mar 2026.

## Placed in

- Multimodal LLMs / vision-language models / [Visual token pruning and efficiency](https://mlsanity.com/mllm/visual-token-pruning-and-efficiency/)

## Builds on (most cited first, 24 of 24)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities](https://mlsanity.com/paper/2507.06261/) (2025)
- [LLaVA-OneVision: Easy Visual Task Transfer](https://mlsanity.com/paper/2408.03326/) (2024)
- [MMBench: Is Your Multi-modal Model an All-around Player?](https://mlsanity.com/paper/2307.06281/) (2023)
- [InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Models](https://mlsanity.com/paper/2504.10479/) (2025)
- [MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Models](https://mlsanity.com/paper/2306.13394/) (2023)
- [Token Merging: Your ViT But Faster](https://mlsanity.com/paper/2210.09461/) (2022)
- [Grounded SAM: Assembling Open-World Models for Diverse Visual Tasks](https://mlsanity.com/paper/2401.14159/) (2024)
- [DynamicViT: Efficient Vision Transformers with Dynamic Token Sparsification](https://mlsanity.com/paper/2106.02034/) (2021)
- [Learned Image Compression with Discretized Gaussian Mixture Likelihoods and Attention Modules](https://mlsanity.com/paper/2001.01568/) (2020)
- [VLMEvalKit: An Open-Source Toolkit for Evaluating Large Multi-Modality Models](https://mlsanity.com/paper/2407.11691/) (2024)
- [CompressAI: a PyTorch library and evaluation platform for end-to-end compression research](https://mlsanity.com/paper/2011.03029/) (2020)
- [TinyCLIP: CLIP Distillation via Affinity Mimicking and Weight Inheritance](https://mlsanity.com/paper/2309.12314/) (2023)
- [VoCo-LLaMA: Towards Vision Compression with Large Language Models](https://mlsanity.com/paper/2406.12275/) (2024)
- [Scalable Image Coding for Humans and Machines](https://mlsanity.com/paper/2107.08373/) (2021)
- [PVC: Progressive Visual Token Compression for Unified Image and Video Processing in Large Vision-Language Models](https://mlsanity.com/paper/2412.09613/) (2024)
- [FocusLLaVA: A Coarse-to-Fine Approach for Efficient and Effective Visual Token Compression](https://mlsanity.com/paper/2411.14228/) (2024)
- [Rate-Distortion Theory in Coding for Machines and its Application](https://mlsanity.com/paper/2305.17295/) (2023)
- [Bridging Compressed Image Latents and Multimodal Large Language Models](https://mlsanity.com/paper/2407.19651/) (2024)
- [High Efficiency Image Compression for Large Visual-Language Models](https://mlsanity.com/paper/2407.17060/) (2024)
- [Privacy-Preserving Autoencoder for Collaborative Object Detection](https://mlsanity.com/paper/2402.18864/) (2024)
- [Tell Codec What Worth Compressing: Semantically Disentangled Image Coding for Machine with LMMs](https://mlsanity.com/paper/2408.08575/) (2024)
- [Compression Beyond Pixels: Semantic Compression with Multimodal Foundation Models](https://mlsanity.com/paper/2509.05925/) (2025)
- [When MLLMs Meet Compression Distortion: A Coding Paradigm Tailored to MLLMs](https://mlsanity.com/paper/2509.24258/) (2025)

## Abstract (arXiv)

The rapid progress of large Vision-Language Models (VLMs) has enabled a wide range of applications, such as image understanding and Visual Question Answering (VQA). Query images are often uploaded to the cloud, where VLMs are typically hosted, hence efficient image compression becomes crucial. However, traditional human-centric codecs are suboptimal in this setting because they preserve many task-irrelevant details. Existing Image Coding for Machines (ICM) methods also fall short, as they assume a fixed set of downstream tasks and cannot adapt to prompt-driven VLMs with an open-ended variety of objectives. We propose a lightweight, plug-and-play, prompt-guided prefiltering module to identify image regions most relevant to the text prompt, and consequently to the downstream task. The module preserves important details while smoothing out less relevant areas to improve compression efficiency. It is codec-agnostic and can be applied before conventional and learned encoders. Experiments on several VQA benchmarks show that our approach achieves a 25-50% average bitrate reduction while maintaining the same task accuracy. Our source code is available at https://github.com/bardia-az/pgp-vlm-compression.