# RUArt: A Novel Text-Centered Solution for Text-Based Visual Question Answering

arXiv: https://arxiv.org/abs/2010.12917 · HTML: https://mlsanity.com/paper/2010.12917/

## Key facts

- Cited by 8 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Oct 2020.

## Built on by (most cited first, 8 of 8)

- [TextOCR: Towards large-scale end-to-end reasoning for arbitrary-shaped scene text](https://mlsanity.com/paper/2105.05486/) (2021)
- [AutoGeo: Automating Geometric Image Dataset Creation for Enhanced Geometry Understanding](https://mlsanity.com/paper/2409.09039/) (2024)
- [Towards Top-Down Reasoning: An Explainable Multi-Agent Approach for Visual Question Answering](https://mlsanity.com/paper/2311.17331/) (2023)
- [Exploring Sparse Spatial Relation in Graph Inference for Text-Based VQA](https://mlsanity.com/paper/2310.09147/) (2023)
- [Scene-Text Grounding for Text-Based Video Question Answering](https://mlsanity.com/paper/2409.14319/) (2024)
- [Make Graph-based Referring Expression Comprehension Great Again through Expression-guided Dynamic Gating and Regression](https://mlsanity.com/paper/2409.03385/) (2024)
- [D2Dewarp: Dual Dimensions Geometric Representation Learning Based Document Image Dewarping](https://mlsanity.com/paper/2507.08492/) (2025)
- [Innocence in the Crossfire: Roles of Skip Connections in Jailbreaking Visual Language Models](https://mlsanity.com/paper/2507.13761/) (2025)

## Abstract (arXiv)

Text-based visual question answering (VQA) requires to read and understand text in an image to correctly answer a given question. However, most current methods simply add optical character recognition (OCR) tokens extracted from the image into the VQA model without considering contextual information of OCR tokens and mining the relationships between OCR tokens and scene objects. In this paper, we propose a novel text-centered method called RUArt (Reading, Understanding and Answering the Related Text) for text-based VQA. Taking an image and a question as input, RUArt first reads the image and obtains text and scene objects. Then, it understands the question, OCRed text and objects in the context of the scene, and further mines the relationships among them. Finally, it answers the related text for the given question through text semantic matching and reasoning. We evaluate our RUArt on two text-based VQA benchmarks (ST-VQA and TextVQA) and conduct extensive ablation studies for exploring the reasons behind RUArt's effectiveness. Experimental results demonstrate that our method can effectively explore the contextual information of the text and mine the stable relationships between the text and objects.