# Finding the Evidence: Localization-aware Answer Prediction for Text Visual Question Answering

arXiv: https://arxiv.org/abs/2010.02582 · HTML: https://mlsanity.com/paper/2010.02582/

## Key facts

- Cited by 19 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 6 Oct 2020.

## Built on by (most cited first, 19 of 19)

- [GIT: A Generative Image-to-text Transformer for Vision and Language](https://mlsanity.com/paper/2205.14100/) (2022)
- [TextOCR: Towards large-scale end-to-end reasoning for arbitrary-shaped scene text](https://mlsanity.com/paper/2105.05486/) (2021)
- [LaTr: Layout-Aware Transformer for Scene-Text VQA](https://mlsanity.com/paper/2112.12494/) (2021)
- [DocFormerv2: Local Features for Document Understanding](https://mlsanity.com/paper/2306.01733/) (2023)
- [PreSTU: Pre-Training for Scene-Text Understanding](https://mlsanity.com/paper/2209.05534/) (2022)
- [Toward 3D Spatial Reasoning for Human-like Text-based Visual Question Answering](https://mlsanity.com/paper/2209.10326/) (2022)
- [Towards Models that Can See and Read](https://mlsanity.com/paper/2301.07389/) (2023)
- [Localize, Group, and Select: Boosting Text-VQA by Scene Text Modeling](https://mlsanity.com/paper/2108.08965/) (2021)
- [TAG: Boosting Text-VQA via Text-aware Visual Question-answer Generation](https://mlsanity.com/paper/2208.01813/) (2022)
- [Separate and Locate: Rethink the Text in Text-based Visual Question Answering](https://mlsanity.com/paper/2308.16383/) (2023)
- [Exploring Sparse Spatial Relation in Graph Inference for Text-Based VQA](https://mlsanity.com/paper/2310.09147/) (2023)
- [MUST-VQA: MUltilingual Scene-text VQA](https://mlsanity.com/paper/2209.06730/) (2022)
- [Locate Then Generate: Bridging Vision and Language with Bounding Box for Scene-Text VQA](https://mlsanity.com/paper/2304.01603/) (2023)
- [Multiple-Question Multiple-Answer Text-VQA](https://mlsanity.com/paper/2311.08622/) (2023)
- [Towards Escaping from Language Bias and OCR Error: Semantics-Centered Text Visual Question Answering](https://mlsanity.com/paper/2203.12929/) (2022)
- [SceneGATE: Scene-Graph based co-Attention networks for TExt visual question answering](https://mlsanity.com/paper/2212.08283/) (2022)
- [EKTVQA: Generalized use of External Knowledge to empower Scene Text in Text-VQA](https://mlsanity.com/paper/2108.09717/) (2021)
- [Towards Multimodal Vision-Language Models Generating Non-Generic Text](https://mlsanity.com/paper/2207.04174/) (2022)
- [Text-Aware Dual Routing Network for Visual Question Answering](https://mlsanity.com/paper/2211.14450/) (2022)

## Abstract (arXiv)

Image text carries essential information to understand the scene and perform reasoning. Text-based visual question answering (text VQA) task focuses on visual questions that require reading text in images. Existing text VQA systems generate an answer by selecting from optical character recognition (OCR) texts or a fixed vocabulary. Positional information of text is underused and there is a lack of evidence for the generated answer. As such, this paper proposes a localization-aware answer prediction network (LaAP-Net) to address this challenge. Our LaAP-Net not only generates the answer to the question but also predicts a bounding box as evidence of the generated answer. Moreover, a context-enriched OCR representation (COR) for multimodal fusion is proposed to facilitate the localization task. Our proposed LaAP-Net outperforms existing approaches on three benchmark datasets for the text VQA task by a noticeable margin.