mlsanityData to 14 Oct 2026

Joint Extraction Matters: Prompt-Based Visual Question Answering for Multi-Field Document Information Extraction

arXiv 2503.16868 ↗First posted 21 Mar 2025cs.CL cs.CVcited by 0

Not on the map. Why?

Abstract (arXiv)

Visual question answering (VQA) has emerged as a flexible approach for extracting specific pieces of information from document images. However, existing work typically queries each field in isolation, overlooking potential dependencies across multiple items. This paper investigates the merits of extracting multiple fields jointly versus separately. Through experiments on multiple large vision language models and datasets, we show that jointly extracting fields often improves accuracy, especially when the fields share strong numeric or contextual dependencies. We further analyze how performance scales with the number of requested items and use a regression based metric to quantify inter field relationships. Our results suggest that multi field prompts can mitigate confusion arising from similar surface forms and related numeric values, providing practical methods for designing robust VQA systems in document information extraction tasks.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 23 of its 26 references that are in the map. First posted on arXiv 21 Mar 2025.

Builds on

23 of 26 references.

References · 23

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2024The Llama 3 Herd of Modelscited by 14,715
  3. 2020End-to-End Object Detection with Transformerscited by 7,414
  4. 2024Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolutioncited by 4,234
  5. 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,795
  6. 2015VQA: Visual Question Answeringcited by 2,506
  7. 2023Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyondcited by 2,224
  8. 2019How Can We Know What Language Models Know?cited by 959
  9. 2019LayoutLM: Pre-training of Text and Layout for Document Image Understandingcited by 553
  10. 2022LayoutLMv3: Pre-training for Document AI with Unified Text and Image Maskingcited by 485
  11. 2020LayoutLMv2: Multi-modal Pre-training for Visually-Rich Document Understandingcited by 408
  12. 2019FUNSD: A Dataset for Form Understanding in Noisy Scanned Documentscited by 349
11 more
  1. 2021ICDAR2019 Competition on Scanned Receipt OCR and Information Extractioncited by 256
  2. 2023Prompting Large Language Model for Machine Translation: A Case Studycited by 250
  3. 2021VisualMRC: Machine Reading Comprehension on Document Imagescited by 151
  4. 2024Does Prompt Formatting Have Any Impact on LLM Performance?cited by 141
  5. 2021LaTr: Layout-Aware Transformer for Scene-Text VQAcited by 72
  6. 2023DocFormerv2: Local Features for Document Understandingcited by 46
  7. 2022IDPG: An Instance-Dependent Prompt Generation Methodcited by 43
  8. 2023Exploring Effectiveness of GPT-3 in Grammatical Error Correction: A Study on Performance and Controllability in Prompt-Based Methodscited by 37
  9. 2022V-Doc : Visual questions answers with Documentscited by 19
  10. 2023Dynamic Prompting: A Unified Framework for Prompt Tuningcited by 12
  11. 2023Multiple-Question Multiple-Answer Text-VQAcited by 4

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2023From Image to Language: A Critical Analysis of Visual Question Answering (VQA) Approaches, Challenges, and Opportunities
  2. 2026Comparative Study of Domain-adapted VLMs for General Document Visual Question Answering
  3. 2024A Comprehensive Survey on Visual Question Answering Datasets and Algorithms
  4. 2024Natural Language Understanding and Inference with MLLM in Visual Question Answering: A Survey
  5. 2025Exploring Advanced Techniques for Visual Question Answering: A Comprehensive Comparison
  6. 2022Towards Complex Document Understanding By Discrete Reasoning
  7. 2025Visual question answering: from early developments to recent advances -- a survey
  8. 2024Detect, Describe, Discriminate: Moving Beyond VQA for MLLM Evaluation