mlsanityData to 14 Oct 2026

ViConsFormer: Constituting Meaningful Phrases of Scene Texts using Transformer-based Method in Vietnamese Text-based Visual Question Answering

arXiv 2410.14132 ↗First posted 18 Oct 2024cs.CV cs.CLcited by 0

Not on the map. Why?

Abstract (arXiv)

Text-based VQA is a challenging task that requires machines to use scene texts in given images to yield the most appropriate answer for the given question. The main challenge of text-based VQA is exploiting the meaning and information from scene texts. Recent studies tackled this challenge by considering the spatial information of scene texts in images via embedding 2D coordinates of their bounding boxes. In this study, we follow the definition of meaning from linguistics to introduce a novel method that effectively exploits the information from scene texts written in Vietnamese. Experimental results show that our proposed method obtains state-of-the-art results on two large-scale Vietnamese Text-based VQA datasets. The implementation can be found at this link.

Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 36 of its 44 references that are in the map. First posted on arXiv 18 Oct 2024.

Builds on

36 of 44 references.

References · 36

  1. 2017Attention Is All You Needcited by 60,361
  2. 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,399
  3. 2014Adam: A Method for Stochastic Optimizationcited by 37,240
  4. 2019Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformercited by 14,599
  5. 2014Neural Machine Translation by Jointly Learning to Align and Translatecited by 4,548
  6. 2016Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answeringcited by 2,516
  7. 2015VQA: Visual Question Answeringcited by 2,506
  8. 2019ViLBERT: Pretraining Task-Agnostic Visiolinguistic Representations for Vision-and-Language Taskscited by 2,222
  9. 2019Towards VQA Models That Can Readcited by 1,718
  10. 2019LXMERT: Learning Cross-Modality Encoder Representations from Transformerscited by 1,553
  11. 2020DocVQA: A Dataset for VQA on Document Imagescited by 1,317
  12. 2019VisualBERT: A Simple and Performant Baseline for Vision and Languagecited by 1,296
24 more
  1. 2015Effective Approaches to Attention-based Neural Machine Translationcited by 1,026
  2. 2019VL-BERT: Pre-training of Generic Visual-Linguistic Representationscited by 954
  3. 2019Unified Vision-Language Pre-Training for Image Captioning and VQAcited by 494
  4. 2019Unicoder-VL: A Universal Encoder for Vision and Language by Cross-modal Pre-trainingcited by 451
  5. 2019Deep Modular Co-Attention Networks for Visual Question Answeringcited by 339
  6. 2019Scene Text Visual Question Answeringcited by 331
  7. 2018Bilinear Attention Networkscited by 280
  8. 2016Hierarchical Question-Image Co-Attention for Visual Question Answeringcited by 278
  9. 2015Stacked Attention Networks for Image Question Answeringcited by 273
  10. 2019Neural Machine Translation with Byte-Level Subwordscited by 115
  11. 2021VinVL: Revisiting Visual Representations in Vision-Language Modelscited by 110
  12. 2019Iterative Answer Prediction with Pointer-Augmented Multimodal Transformers for TextVQAcited by 107
  13. 2021LaTr: Layout-Aware Transformer for Scene-Text VQAcited by 72
  14. 2022SwinTextSpotter: Scene Text Spotting via Better Synergy between Text Detection and Text Recognitioncited by 70
  15. 2020X-LXMERT: Paint, Caption and Answer Questions with Multi-Modal Transformerscited by 59
  16. 2020Spatially Aware Multimodal Transformers for TextVQAcited by 44
  17. 2022ViT5: Pretrained Text-to-Text Transformer for Vietnamese Language Generationcited by 37
  18. 2022PreSTU: Pre-Training for Scene-Text Understandingcited by 27
  19. 2019Compact Trilinear Interaction for Visual Question Answeringcited by 24
  20. 2020Structured Multimodal Attentions for TextVQAcited by 23
  21. 2023OpenViVQA: Task, Dataset, and Multimodal Fusion Models for Visual Question Answering in Vietnamesecited by 20
  22. 2024ViOCRVQA: Novel Benchmark Dataset and Vision Reader for Visual Question Answering by Understanding Vietnamese Text in Imagescited by 8
  23. 2024ViTextVQA: A Large-Scale Visual Question Answering Dataset and a Novel Multimodal Feature Fusion Method for Vietnamese Text Comprehension in Imagescited by 7
  24. 2023ViCLEVR: A Visual Reasoning Dataset and Hybrid Multimodal Fusion Model for Visual Question Answering in Vietnamesecited by 5

Built on by

Not cited yet.