# Decoupling semantics from vision: A framework for faithful visual-text compression evaluation

arXiv: https://arxiv.org/abs/2608.01848 · HTML: https://mlsanity.com/paper/2608.01848/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in Visual token pruning in multimodal models, one of 12 directions in Efficient LLM inference and serving.
- Builds on 30 of its 35 references that are in the map.
- First posted on arXiv 3 Aug 2026.

## Placed in

- Efficient LLM inference and serving / [Visual token pruning in multimodal models](https://mlsanity.com/inference/visual-token-pruning-in-multimodal-models/)

## Builds on (most cited first, 30 of 30)

- [Qwen2.5-VL Technical Report](https://mlsanity.com/paper/2502.13923/) (2025)
- [LongBench: A Bilingual, Multitask Benchmark for Long Context Understanding](https://mlsanity.com/paper/2308.14508/) (2023)
- [RULER: What's the Real Context Size of Your Long-Context Language Models?](https://mlsanity.com/paper/2404.06654/) (2024)
- [OCRBench: On the Hidden Mystery of OCR in Large Multimodal Models](https://mlsanity.com/paper/2305.07895/) (2023)
- [OCR-free Document Understanding Transformer](https://mlsanity.com/paper/2111.15664/) (2021)
- [Pix2Struct: Screenshot Parsing as Pretraining for Visual Language Understanding](https://mlsanity.com/paper/2210.03347/) (2022)
- [MiniMax-M1: Scaling Test-Time Compute Efficiently with Lightning Attention](https://mlsanity.com/paper/2506.13585/) (2025)
- [DeepSeek-OCR: Contexts Optical Compression](https://mlsanity.com/paper/2510.18234/) (2025)
- [General OCR Theory: Towards OCR-2.0 via a Unified End-to-end Model](https://mlsanity.com/paper/2409.01704/) (2024)
- [OmniDocBench: Benchmarking Diverse PDF Document Parsing with Comprehensive Annotations](https://mlsanity.com/paper/2412.07626/) (2024)
- [Language Modelling with Pixels](https://mlsanity.com/paper/2207.06991/) (2022)
- [Revisiting the Role of Language Priors in Vision-Language Models](https://mlsanity.com/paper/2306.01879/) (2023)
- [Glyph: Scaling Context Windows via Visual-Text Compression](https://mlsanity.com/paper/2510.17800/) (2025)
- [Vision-centric Token Compression in Large Language Model](https://mlsanity.com/paper/2502.00791/) (2025)
- [Tokenization Falling Short: On Subword Robustness in Large Language Models](https://mlsanity.com/paper/2406.11687/) (2024)
- [Render-of-Thought: Rendering Textual Chain-of-Thought as Images for Visual Latent Reasoning](https://mlsanity.com/paper/2601.14750/) (2026)
- [Leveraging Visual Tokens for Extended Text Contexts in Multi-Modal Learning](https://mlsanity.com/paper/2406.02547/) (2024)
- [Long Context is Not Long at All: A Prospector of Long-Dependency Data for Large Language Models](https://mlsanity.com/paper/2405.17915/) (2024)
- [One Leaf Reveals the Season: Occlusion-Based Contrastive Learning with Semantic-Aware Views for Efficient Visual Representation](https://mlsanity.com/paper/2411.09858/) (2024)
- [Fewer Tokens, Greater Scaling: Self-Adaptive Visual Bases for Efficient and Expansive Representation Learning](https://mlsanity.com/paper/2511.19515/) (2025)
- [Multilingual Pretraining for Pixel Language Models](https://mlsanity.com/paper/2505.21265/) (2025)
- [VTCBench: Can Vision-Language Models Understand Long Context with Vision-Text Compression?](https://mlsanity.com/paper/2512.15649/) (2025)
- [The Model Knows Which Tokens Matter: Automatic Token Selection via Noise Gating](https://mlsanity.com/paper/2603.07135/) (2026)
- [ZeroSense:How Vision matters in Long Context Compression](https://mlsanity.com/paper/2603.11846/) (2026)
- [See the Text: From Tokenization to Visual Reading](https://mlsanity.com/paper/2510.18840/) (2025)
- [Visual Merit or Linguistic Crutch? A Close Look at DeepSeek-OCR](https://mlsanity.com/paper/2601.03714/) (2026)
- [VTC-R1: Vision-Text Compression for Efficient Long-Context Reasoning](https://mlsanity.com/paper/2601.22069/) (2026)
- [Beyond Surrogate Gradients: Fully Differentiable Token Pruning for Vision-Language Models](https://mlsanity.com/paper/2605.28051/) (2026)
- [Stepwise Token Selection for Efficient Multimodal Large Language Models](https://mlsanity.com/paper/2606.16067/) (2026)
- [LensVLM: Selective Context Expansion for Compressed Visual Representation of Text](https://mlsanity.com/paper/2605.07019/) (2026)

## Built on by (most cited first, 2 of 2)

- [StepPrune: Adaptive Sequential Visual Token Selection across Multimodal Large Language Models](https://mlsanity.com/paper/2609.13804/) (2026)
- [FocusVTC: Efficient and High-Performance Visual Text Compression with Adaptive Resolution](https://mlsanity.com/paper/2609.36651/) (2026)

## Abstract (arXiv)

Recent visual-text compression (VTC) methods, typified by DeepSeek-OCR, report impressive high token compression ratios for long-context modeling tasks by leveraging text-to-image rendering. However, existing evaluation protocols heavily rely on downstream task performance. Such evaluation metrics fail to accurately measure text preservation due to the strong inherent linguistic priors of Multimodal Large Language Models (MLLMs). In this work, we introduce a new evaluation framework that decouples MLLMs' capabilities to faithfully assess VTC quality. Within this framework, we further introduce the ZeroSense Benchmark to ensure low semantic correlation of testing samples. By eliminating textual dependencies, our benchmark guarantees that the evaluation results are purely reflective of VTC quality, unaffected by the semantic inference capabilities of downstream models. Extensive experiments across multiple datasets demonstrate that VTC quality and downstream task accuracy diverge significantly, highlighting the necessity of our decoupled evaluation framework.