# P-MTP: Efficient Document Parsing via Multi-Token Prediction with Progressive Depth Scaling

arXiv: https://arxiv.org/abs/2606.24447 · HTML: https://mlsanity.com/paper/2606.24447/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Placed in Document understanding and multimodal RAG, one of 16 directions in Multimodal LLMs / vision-language models.
- Builds on 23 of its 26 references that are in the map.
- First posted on arXiv 23 Jun 2026.

## Placed in

- Multimodal LLMs / vision-language models / [Document understanding and multimodal RAG](https://mlsanity.com/mllm/document-understanding-and-multimodal-rag/)

## Builds on (most cited first, 23 of 23)

- [Qwen3 Technical Report](https://mlsanity.com/paper/2505.09388/) (2025)
- [Efficient Memory Management for Large Language Model Serving with PagedAttention](https://mlsanity.com/paper/2309.06180/) (2023)
- [Accelerating Large Language Model Decoding with Speculative Sampling](https://mlsanity.com/paper/2302.01318/) (2023)
- [Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Heads](https://mlsanity.com/paper/2401.10774/) (2024)
- [EAGLE: Speculative Sampling Requires Rethinking Feature Uncertainty](https://mlsanity.com/paper/2401.15077/) (2024)
- [8-bit Optimizers via Block-wise Quantization](https://mlsanity.com/paper/2110.02861/) (2021)
- [EAGLE-2: Faster Inference of Language Models with Dynamic Draft Trees](https://mlsanity.com/paper/2406.16858/) (2024)
- [Better & Faster Large Language Models via Multi-token Prediction](https://mlsanity.com/paper/2404.19737/) (2024)
- [EAGLE-3: Scaling up Inference Acceleration of Large Language Models via Training-Time Test](https://mlsanity.com/paper/2503.01840/) (2025)
- [MinerU: An Open-Source Solution for Precise Document Content Extraction](https://mlsanity.com/paper/2409.18839/) (2024)
- [Image-based table recognition: data, model, and evaluation](https://mlsanity.com/paper/1911.10683/) (2019)
- [OmniDocBench: Benchmarking Diverse PDF Document Parsing with Comprehensive Annotations](https://mlsanity.com/paper/2412.07626/) (2024)
- [PaddleOCR 3.0 Technical Report](https://mlsanity.com/paper/2507.05595/) (2025)
- [Hydra: Sequentially-Dependent Draft Heads for Medusa Decoding](https://mlsanity.com/paper/2402.05109/) (2024)
- [Dolphin: Document Image Parsing via Heterogeneous Anchor Prompting](https://mlsanity.com/paper/2505.14059/) (2025)
- [DeepSeek-OCR 2: Visual Causal Flow](https://mlsanity.com/paper/2601.20552/) (2026)
- [Document Parsing Unveiled: Techniques, Challenges, and Prospects for Structured Information Extraction](https://mlsanity.com/paper/2410.21169/) (2024)
- [LightOnOCR: A 1B End-to-End Multilingual Vision-Language Model for State-of-the-Art OCR](https://mlsanity.com/paper/2601.14251/) (2026)
- [Dolphin-v2: Universal Document Parsing via Scalable Anchor Prompting](https://mlsanity.com/paper/2602.05384/) (2026)
- [Youtu-Parsing: Perception, Structuring and Recognition via High-Parallelism Decoding](https://mlsanity.com/paper/2601.20430/) (2026)
- [Gumiho: A Hybrid Architecture to Prioritize Early Tokens in Speculative Decoding](https://mlsanity.com/paper/2503.10135/) (2025)
- [Amphista: Bi-directional Multi-head Decoding for Accelerating LLM Inference](https://mlsanity.com/paper/2406.13170/) (2024)
- [UniRec: Unified Multimodal Encoding for LLM-Based Recommendations](https://mlsanity.com/paper/2601.19423/) (2026)

## Built on by (most cited first, 2 of 2)

- [HPD-Parsing: Hierarchical Parallel Document Parsing](https://mlsanity.com/paper/2607.18839/) (2026)
- [HyperZip: Efficient Data Compression through Personalized Diffusion LLMs with Hypernetworks](https://mlsanity.com/paper/2609.36357/) (2026)

## Abstract (arXiv)

Vision-Language Models (VLMs) have revolutionized document parsing by enabling end-to-end mapping from images to structured text, imposing a significant latency bottleneck, particularly for token-dense documents. While Multi-Token Prediction (MTP) has emerged as a promising approach for accelerating inference, its potential is constrained by optimization instability when scaling to deeper look-ahead depth. In this paper, we propose P-MTP, a framework that leverages Progressive Multi-Token Prediction with a lightweight MTP module to scale the look-ahead depth for high-throughput document parsing. Specifically, we introduce Progressive Curriculum Loss that adaptively re-weights different look-ahead depths using cumulative path reliability and retrospective target consistency. By effectively suppressing gradient noise in long-range predictions, P-MTP, facilitates an automated easy-to-hard optimization transition, enabling the model to master increasingly distant look-ahead depths. Furthermore, we propose Confidence-Gated Dynamic Drafting to maximize the effective look-ahead depth and acceptance rate by adaptively calibrating speculative length during inference, thereby minimizing computational waste and further pushing the boundaries of inference speedup. Experimental results across multiple benchmarks and architectures demonstrate that P-MTP, achieves up to a 5× speedup with negligible loss in accuracy, providing the first successful validation of extensive look-ahead MTP in the document parsing domain.