mlsanityData to 7 Oct 2026

ViTextVQA: A Large-Scale Visual Question Answering Dataset and a Novel Multimodal Feature Fusion Method for Vietnamese Text Comprehension in Images

arXiv 2404.10652 ↗First posted 16 Apr 2024cs.CLcited by 7

Not on the map. Why?

Abstract (arXiv)

Visual Question Answering (VQA) is a challenging task that requires the joint understanding of natural language and visual content. While early research primarily focused on recognizing objects and scene context, it often overlooked scene text-an essential source of explicit semantic information. This paper introduces ViTextVQA (Vietnamese Text-based Visual Question Answering), the first large-scale Vietnamese dataset specializing in text-based VQA. The dataset contains over 16,000 images and over 50,000 question-answer pairs. To tackle this task efficiently, ViTextBLIP-2 (Vietnamese Text-based Bootstrapped Language-Image Model via Fine-tuning) is proposed, a novel multimodal feature fusion method designed to optimize Vietnamese text-based VQA. Experiments with state-of-the-art models highlight the importance of token ordering in OCR text for answer generation, leading to significant performance improvements. The ViTextVQA dataset is publicly available for research purposes.

Key facts. Cited by 7 papers in the map (data through 7 Oct 2026). Builds on 54 of its 82 references that are in the map. First posted on arXiv 16 Apr 2024.

Builds on

54 of 82 references.

References · 54

  1. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
  3. 2014Adam: A Method for Stochastic Optimizationcited by 36,960
  4. 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
  5. 2023GPT-4 Technical Reportcited by 18,132
  6. 2014Very Deep Convolutional Networks for Large-Scale Image Recognitioncited by 15,267
  7. 2023BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Modelscited by 6,069
  8. 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,337
  9. 2022BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generationcited by 4,310
  10. 2014Going Deeper with Convolutionscited by 4,231
  11. 2016SQuAD: 100,000+ Questions for Machine Comprehension of Textcited by 4,115
  12. 2016Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answeringcited by 2,498
38 more
  1. 2015VQA: Visual Question Answeringcited by 2,489
  2. 2014Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modelingcited by 2,341
  3. 2021Learning to Prompt for Vision-Language Modelscited by 2,269
  4. 2019ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Taskscited by 2,219
  5. 2023Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyondcited by 2,179
  6. 2013Rich feature hierarchies for accurate object detection and semantic segmentationcited by 2,160
  7. 2019Towards VQA Models That Can Readcited by 1,696
  8. 2022Visual Prompt Tuningcited by 1,674
  9. 2019LXMERT: Learning Cross-Modality Encoder Representations from Transformerscited by 1,548
  10. 2017Bottom-Up and Top-Down Attention for Image Captioning and Visual Question Answeringcited by 1,310
  11. 2020DocVQA: A Dataset for VQA on Document Imagescited by 1,288
  12. 2020Oscar: Object-Semantics Aligned Pre-training for Vision-Language Taskscited by 1,213
  13. 2019OK-VQA: A Visual Question Answering Benchmark Requiring External Knowledgecited by 1,114
  14. 2019VL-BERT: Pre-training of Generic Visual-Linguistic Representationscited by 954
  15. 2018VizWiz Grand Challenge: Answering Visual Questions from Blind Peoplecited by 852
  16. 2019How multilingual is Multilingual BERT?cited by 642
  17. 2022GIT: A Generative Image-to-text Transformer for Vision and Languagecited by 536
  18. 2019Unicoder-VL: A Universal Encoder for Vision and Language by Cross-modal Pre-trainingcited by 451
  19. 2021InfographicVQAcited by 430
  20. 2019Scene Text Visual Question Answeringcited by 327
  21. 2016CNN-RNN: A Unified Framework for Multi-label Image Classificationcited by 189
  22. 2022mPLUG: Effective and Efficient Vision-Language Learning by Cross-modal Skip-connectionscited by 185
  23. 2021VisualMRC: Machine Reading Comprehension on Document Imagescited by 150
  24. 2014A Multi-World Approach to Question Answering about Real-World Scenes based on Uncertain Inputcited by 112
  25. 2019Iterative Answer Prediction with Pointer-Augmented Multimodal Transformers for TextVQAcited by 107
  26. 2015Are You Talking to a Machine? Dataset and Methods for Multilingual Image Question Answeringcited by 80
  27. 2021LaTr: Layout-Aware Transformer for Scene-Text VQAcited by 72
  28. 2022SwinTextSpotter: Scene Text Spotting via Better Synergy between Text Detection and Text Recognitioncited by 70
  29. 2015Ask, Attend and Answer: Exploring Question-Guided Spatial Attention for Visual Question Answeringcited by 66
  30. 2018VnCoreNLP: A Vietnamese Natural Language Processing Toolkitcited by 51
  31. 2023mBLIP: Efficient Bootstrapping of Multilingual Vision-LLMscited by 41
  32. 2022ViT5: Pretrained Text-to-Text Transformer for Vietnamese Language Generationcited by 37
  33. 2022PreSTU: Pre-Training for Scene-Text Understandingcited by 27
  34. 2017A Fast and Accurate Vietnamese Word Segmentercited by 21
  35. 2023OpenViVQA: Task, Dataset, and Multimodal Fusion Models for Visual Question Answering in Vietnamesecited by 20
  36. 2016Automatic Generation of Grounded Visual Questionscited by 19
  37. 2023ViHOS: Hate Speech Spans Detection for Vietnamesecited by 13
  38. 2023Separate and Locate: Rethink the Text in Text-based Visual Question Answeringcited by 7

50 most cited of 54 All 54 (JSON)

Built on by

Cited by year

Citing papers per year: 2024: 3; 2025: 4.

7 papers.

  1. 2024Vintern-1B: An Efficient Multimodal Large Language Model for Vietnamesecited by 12
  2. 2025Pearl: A Multimodal Culturally-Aware Arabic Instruction Datasetcited by 6
  3. 2024Reference-Based Post-OCR Processing with LLM for Precise Diacritic Text in Historical Document Recognitioncited by 5
  4. 2025Evaluation of Cultural Competence of Vision-Language Modelscited by 3
  5. 2024ViConsFormer: Constituting Meaningful Phrases of Scene Texts using Transformer-based Method in Vietnamese Text-based Visual Question Answeringcited by 0
  6. 2025ViInfographicVQA: A Benchmark for Single and Multi-image Visual Question Answering on Vietnamese Infographicscited by 0
  7. 2025Towards Signboard-Oriented Visual Question Answering: ViSignVQA Dataset, Method and Benchmarkcited by 0