mlsanityData to 14 Oct 2026

Text-VQA Aug: Pipelined Harnessing of Large Multimodal Models for Automated Synthesis

arXiv 2511.02046 ↗First posted 3 Nov 2025cs.CV cs.AIcited by 0

On the map: Multimodal LLMs / vision-language models → VLM benchmarks and visual reasoning

Abstract (arXiv)

Creation of large-scale databases for Visual Question Answering tasks pertaining to the text data in a scene (text-VQA) involves skilful human annotation, which is tedious and challenging. With the advent of foundation models that handle vision and language modalities, and with the maturity of OCR systems, it is the need of the hour to establish an end-to-end pipeline that can synthesize Question-Answer (QA) pairs based on scene-text from a given image. We propose a pipeline for automated synthesis for text-VQA dataset that can produce faithful QA pairs, and which scales up with the availability of scene text data. Our proposed method harnesses the capabilities of multiple models and algorithms involving OCR detection and recognition (text spotting), region of interest (ROI) detection, caption generation, and question generation. These components are streamlined into a cohesive pipeline to automate the synthesis and validation of QA pairs. To the best of our knowledge, this is the first pipeline proposed to automatically synthesize and validate a large-scale text-VQA dataset comprising around 72K QA pairs based on around 44K images.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Placed in VLM benchmarks and visual reasoning, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 23 of its 37 references that are in the map. First posted on arXiv 3 Nov 2025.

Builds on

23 of 37 references.

References · 23

  1. 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
  2. 2023Judging LLM-as-a-Judge with MT-Bench and Chatbot Arenacited by 8,626
  3. 2023Direct Preference Optimization: Your Language Model is Secretly a Reward Modelcited by 8,065
  4. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,968
  5. 2023Improved Baselines with Visual Instruction Tuningcited by 4,418
  6. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,148
  7. 2016Visual Genome: Connecting Language and Vision Using Crowdsourced Dense Image Annotationscited by 2,656
  8. 2015VQA: Visual Question Answeringcited by 2,506
  9. 2021BEiT: BERT Pre-Training of Image Transformerscited by 2,375
  10. 2019Towards VQA Models That Can Readcited by 1,718
  11. 2018VizWiz Grand Challenge: Answering Visual Questions from Blind Peoplecited by 858
  12. 2021Florence: A New Foundation Model for Computer Visioncited by 804
11 more
  1. 2019Scene Text Visual Question Answeringcited by 331
  2. 2015Visual7W: Grounded Question Answering in Imagescited by 318
  3. 2017EAST: An Efficient and Accurate Scene Text Detectorcited by 248
  4. 2016COCO-Text: Dataset and Benchmark for Text Detection and Recognition in Natural Imagescited by 236
  5. 2022Scene Text Recognition with Permuted Autoregressive Sequence Modelscited by 149
  6. 2015Exploring Models and Data for Image Question Answeringcited by 139
  7. 2018TextBoxes++: A Single-Shot Oriented Scene Text Detectorcited by 119
  8. 2020TAP: Text-Aware Pre-training for Text-VQA and Text-Captioncited by 95
  9. 2021Open Images V5 Text Annotation and Yet Another Mask Text Spottercited by 46
  10. 2023CLIP4STR: A Simple Baseline for Scene Text Recognition with Pre-trained Vision-Language Modelcited by 30
  11. 2018Detecting Multi-Oriented Text with Corner-based Region Proposalscited by 3

Built on by

Not cited yet.

Similar, not cited

Close in wording, no citation either way

  1. 2025Exploring Advanced Techniques for Visual Question Answering: A Comprehensive Comparison
  2. 2026Beyond OCR Accuracy: Text-Centric VQA Under Image Degradation with Modular and End-to-End
  3. 2024A Comprehensive Survey on Visual Question Answering Datasets and Algorithms
  4. 2025Visual question answering: from early developments to recent advances -- a survey
  5. 2023From Image to Language: A Critical Analysis of Visual Question Answering (VQA) Approaches, Challenges, and Opportunities
  6. 2024MTVQA: Benchmarking Multilingual Text-Centric Visual Question Answering
  7. 2024Natural Language Understanding and Inference with MLLM in Visual Question Answering: A Survey
  8. 2023What Large Language Models Bring to Text-rich VQA?