# Towards Escaping from Language Bias and OCR Error: Semantics-Centered Text Visual Question Answering

arXiv: https://arxiv.org/abs/2203.12929 · HTML: https://mlsanity.com/paper/2203.12929/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 17 of its 21 references that are in the map.
- First posted on arXiv 24 Mar 2022.

## Builds on (most cited first, 17 of 17)

- [Attention Is All You Need](https://mlsanity.com/paper/1706.03762/) (2017)
- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://mlsanity.com/paper/1506.01497/) (2015)
- [Making the V in VQA Matter: Elevating the Role of Image Understanding in Visual Question Answering](https://mlsanity.com/paper/1612.00837/) (2016)
- [Towards VQA Models That Can Read](https://mlsanity.com/paper/1904.08920/) (2019)
- [Enriching Word Vectors with Subword Information](https://mlsanity.com/paper/1607.04606/) (2016)
- [Bottom-Up and Top-Down Attention for Image Captioning and Visual Question Answering](https://mlsanity.com/paper/1707.07998/) (2017)
- [Scene Text Visual Question Answering](https://mlsanity.com/paper/1905.13648/) (2019)
- [Iterative Answer Prediction with Pointer-Augmented Multimodal Transformers for TextVQA](https://mlsanity.com/paper/1911.06258/) (2019)
- [TAP: Text-Aware Pre-training for Text-VQA and Text-Caption](https://mlsanity.com/paper/2012.04638/) (2020)
- [On the General Value of Evidence, and Bilingual Scene-Text Visual Question Answering](https://mlsanity.com/paper/2002.10215/) (2020)
- [Pythia v0.1: the Winning Entry to the VQA Challenge 2018](https://mlsanity.com/paper/1807.09956/) (2018)
- [Spatially Aware Multimodal Transformers for TextVQA](https://mlsanity.com/paper/2007.12146/) (2020)
- [Omnidirectional Scene Text Detection with Sequential-free Box Discretization](https://mlsanity.com/paper/1906.02371/) (2019)
- [Structured Multimodal Attentions for TextVQA](https://mlsanity.com/paper/2006.00753/) (2020)
- [Finding the Evidence: Localization-aware Answer Prediction for Text Visual Question Answering](https://mlsanity.com/paper/2010.02582/) (2020)
- [Simple is not Easy: A Simple Strong Baseline for TextVQA and TextCaps](https://mlsanity.com/paper/2012.05153/) (2020)

## Built on by (most cited first, 2 of 2)

- [DocFormerv2: Local Features for Document Understanding](https://mlsanity.com/paper/2306.01733/) (2023)
- [Separate and Locate: Rethink the Text in Text-based Visual Question Answering](https://mlsanity.com/paper/2308.16383/) (2023)

## Abstract (arXiv)

Texts in scene images convey critical information for scene understanding and reasoning. The abilities of reading and reasoning matter for the model in the text-based visual question answering (TextVQA) process. However, current TextVQA models do not center on the text and suffer from several limitations. The model is easily dominated by language biases and optical character recognition (OCR) errors due to the absence of semantic guidance in the answer prediction process. In this paper, we propose a novel Semantics-Centered Network (SC-Net) that consists of an instance-level contrastive semantic prediction module (ICSP) and a semantics-centered transformer module (SCT). Equipped with the two modules, the semantics-centered model can resist the language biases and the accumulated errors from OCR. Extensive experiments on TextVQA and ST-VQA datasets show the effectiveness of our model. SC-Net surpasses previous works with a noticeable margin and is more reasonable for the TextVQA task.