# Multimodal grid features and cell pointers for Scene Text Visual Question Answering

arXiv: https://arxiv.org/abs/2006.00923 · HTML: https://mlsanity.com/paper/2006.00923/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 1 Jun 2020.

## Built on by (most cited first, 5 of 5)

- [Hierarchical multimodal transformers for Multi-Page DocVQA](https://mlsanity.com/paper/2212.05935/) (2022)
- [LaTr: Layout-Aware Transformer for Scene-Text VQA](https://mlsanity.com/paper/2112.12494/) (2021)
- [OCR-IDL: OCR Annotations for Industry Document Library Dataset](https://mlsanity.com/paper/2202.12985/) (2022)
- [MUST-VQA: MUltilingual Scene-text VQA](https://mlsanity.com/paper/2209.06730/) (2022)
- [Spatially Grounded Explanations in Vision Language Models for Document Visual Question Answering](https://mlsanity.com/paper/2507.12490/) (2025)

## Abstract (arXiv)

This paper presents a new model for the task of scene text visual question answering, in which questions about a given image can only be answered by reading and understanding scene text that is present in it. The proposed model is based on an attention mechanism that attends to multi-modal features conditioned to the question, allowing it to reason jointly about the textual and visual modalities in the scene. The output weights of this attention module over the grid of multi-modal spatial features are interpreted as the probability that a certain spatial location of the image contains the answer text the to the given question. Our experiments demonstrate competitive performance in two standard datasets. Furthermore, this paper provides a novel analysis of the ST-VQA dataset based on a human performance study.