mlsanityData to 7 Oct 2026

Up to 36x Speedup: Mask-based Parallel Inference Paradigm for Key Information Extraction in MLLMs

arXiv 2601.19613 ↗First posted 27 Jan 2026cs.CL cs.AIcited by 0

On the map: Multimodal LLMs / vision-language models → Document understanding and multimodal RAG

Abstract (arXiv)

Key Information Extraction (KIE) from visually-rich documents (VrDs) is a critical task, for which recent Large Language Models (LLMs) and Multi-Modal Large Language Models (MLLMs) have demonstrated strong potential. However, their reliance on autoregressive inference, which generates outputs sequentially, creates a significant efficiency bottleneck, especially as KIE tasks often involve extracting multiple, semantically independent fields. To overcome this limitation, we introduce PIP: a Parallel Inference Paradigm for KIE. Our approach reformulates the problem by using "[mask]" tokens as placeholders for all target values, enabling their simultaneous generation in a single forward pass. To facilitate this paradigm, we develop a tailored mask pre-training strategy and construct large-scale supervised datasets. Experimental results show that our PIP-models achieve a 5-36x inference speedup with negligible performance degradation compared to traditional autoregressive base models. By substantially improving efficiency while maintaining high accuracy, PIP paves the way for scalable and practical real-world KIE solutions.

Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Document understanding and multimodal RAG, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 21 of its 27 references that are in the map. First posted on arXiv 27 Jan 2026.

Builds on

21 of 27 references.

Closest ancestors · 2

Same direction, newest first

  1. 2024LayoutLLM: Layout Instruction Tuning with Large Language Models for Document Understanding8 Apr
  2. 2024TextMonkey: An OCR-Free Large Multimodal Model for Understanding Document7 Mar

Other references · 19

  1. 2023GPT-4 Technical Reportcited by 18,132
  2. 2023LLaMA: Open and Efficient Foundation Language Modelscited by 13,958
  3. 2023Llama 2: Open Foundation and Fine-Tuned Chat Modelscited by 12,865
  4. 2023Visual Instruction Tuningcited by 7,792
  5. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,151
  6. 2024Gemini 1.5: Unlocking multimodal understanding across millions of tokens of contextcited by 3,175
  7. 2023MiniGPT-4: Enhancing Vision-Language Understanding with Advanced Large Language Modelscited by 2,689
  8. 2024How Far Are We to GPT-4V? Closing the Gap to Commercial Multimodal Models with Open-Source Suitescited by 1,033
  9. 2025Large Language Diffusion Modelscited by 872
  10. 2024DeepSeek-VL: Towards Real-World Vision-Language Understandingcited by 718
  11. 2025GLM-4.5: Agentic, Reasoning, and Coding (ARC) Foundation Modelscited by 446
  12. 2019FUNSD: A Dataset for Form Understanding in Noisy Scanned Documentscited by 345
7 more
  1. 2023LLaVAR: Enhanced Visual Instruction Tuning for Text-Rich Image Understandingcited by 272
  2. 2021ICDAR2019 Competition on Scanned Receipt OCR and Information Extractioncited by 254
  3. 2025DeepSeek-OCR: Contexts Optical Compressioncited by 174
  4. 2024A Bounding Box is Worth One Token: Interleaving Layout and Text in a Large Language Model for Document Understandingcited by 57
  5. 2023Visual Information Extraction in the Wild: Practical Dataset and End-to-end Solutioncited by 54
  6. 2021Spatial Dual-Modality Graph Reasoning for Key Information Extractioncited by 43
  7. 2024PaDeLLM-NER: Parallel Decoding in Large Language Models for Named Entity Recognitioncited by 6

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Mask-Enhanced Autoregressive Prediction: Pay Less Attention to Learn More
  2. 2026MM-ShiftKV: Decode-Aware Prefill-Stage KV Selection for Multimodal Large Language Models
  3. 2026Efficient Document Parsing via Parallel Token Prediction
  4. 2026UNIKIE-BENCH: Benchmarking Large Multimodal Models for Key Information Extraction in Visual Documents
  5. 2026Enhancing Large Multimodal Models in Key Information Extraction via Scene-Aware Document Synthesis
  6. 2026MiCo: Mutual Information Coverage Optimization through Semantic Erasure Modeling for Efficient MLLM Inference
  7. 2025VDInstruct: Zero-Shot Key Information Extraction via Content-Aware Vision Tokenization
  8. 2024Boosting Multimodal Large Language Models with Visual Tokens Withdrawal for Rapid Inference