# ZeroSense:How Vision matters in Long Context Compression

arXiv: https://arxiv.org/abs/2603.11846 · HTML: https://mlsanity.com/paper/2603.11846/

## Key facts

- Cited by 11 papers in the map (data through 14 Oct 2026).
- Placed in Visual token pruning in multimodal models, one of 12 directions in Efficient LLM inference and serving.
- First posted on arXiv 12 Mar 2026.

## Placed in

- Efficient LLM inference and serving / [Visual token pruning in multimodal models](https://mlsanity.com/inference/visual-token-pruning-in-multimodal-models/)

## Built on by (most cited first, 11 of 11)

- [Beyond Surrogate Gradients: Fully Differentiable Token Pruning for Vision-Language Models](https://mlsanity.com/paper/2605.28051/) (2026)
- [Multimodal Model for Computational Pathology:Representation Learning and Image Compression](https://mlsanity.com/paper/2603.18660/) (2026)
- [A unified multi-task framework enables interpretable chest radiograph analysis](https://mlsanity.com/paper/2606.03417/) (2026)
- [Stepwise Token Selection for Efficient Multimodal Large Language Models](https://mlsanity.com/paper/2606.16067/) (2026)
- [Learnable Token Sparsification for Efficient Gigapixel Whole Slide Image Reasoning](https://mlsanity.com/paper/2606.08641/) (2026)
- [LensVLM: Selective Context Expansion for Compressed Visual Representation of Text](https://mlsanity.com/paper/2605.07019/) (2026)
- [Decoupling semantics from vision: A framework for faithful visual-text compression evaluation](https://mlsanity.com/paper/2608.01848/) (2026)
- [Same Semantics, Different Paths: Self-Improving Alignment for Vision-Text Compression](https://mlsanity.com/paper/2608.02109/) (2026)
- [DiffPrune: differentiable information throttling for token pruning in vision-language models](https://mlsanity.com/paper/2608.01985/) (2026)
- [From Snapshots to Symphonies: The Evolution of Protein Prediction from Static Structures to Generative Dynamics and Multimodal Interactions](https://mlsanity.com/paper/2603.18505/) (2026)
- [StepPrune: Adaptive Sequential Visual Token Selection across Multimodal Large Language Models](https://mlsanity.com/paper/2609.13804/) (2026)

## Abstract (arXiv)

Recent visual-text compression (VTC) methods, typified by DeepSeek-OCR, report impressive high token compression ratios for long-context modeling tasks by leveraging text-to-image rendering. However, existing evaluation protocols heavily rely on downstream task performance. Such evaluation metrics fail to accurately measure text preservation due to the strong inherent linguistic priors of Multimodal Large Language Models (MLLMs). In this work, we introduce a new evaluation framework that decouples MLLMs' capabilities to faithfully assess VTC quality. Within this framework, we further introduce the ZeroSense Benchmark to ensure low semantic correlation of testing samples. By eliminating contextual dependencies, our benchmark guarantees that the evaluation results are purely reflective of VTC quality, unaffected by the semantic inference capabilities of downstream models. Extensive experiments across multiple datasets demonstrate that VTC quality and downstream task accuracy diverge significantly, highlighting the necessity of our decoupled evaluation framework.