# ICDAR 2019 Competition on Scene Text Visual Question Answering

arXiv: https://arxiv.org/abs/1907.00490 · HTML: https://mlsanity.com/paper/1907.00490/

## Key facts

- Cited by 46 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 30 Jun 2019.

## Built on by (most cited first, 30 of 46)

- [OCRBench: On the Hidden Mystery of OCR in Large Multimodal Models](https://mlsanity.com/paper/2305.07895/) (2023)
- [LLaVAR: Enhanced Visual Instruction Tuning for Text-Rich Image Understanding](https://mlsanity.com/paper/2306.17107/) (2023)
- [LVLM-eHub: A Comprehensive Evaluation Benchmark for Large Vision-Language Models](https://mlsanity.com/paper/2306.09265/) (2023)
- [DePlot: One-shot visual language reasoning by plot-to-table translation](https://mlsanity.com/paper/2212.10505/) (2022)
- [ImageBind-LLM: Multi-modality Instruction Tuning](https://mlsanity.com/paper/2309.03905/) (2023)
- [Document Understanding Dataset and Evaluation (DUDE)](https://mlsanity.com/paper/2305.08455/) (2023)
- [Going Full-TILT Boogie on Document Understanding with Text-Image-Layout Transformer](https://mlsanity.com/paper/2102.09550/) (2021)
- [DocLLM: A layout-aware generative language model for multimodal document understanding](https://mlsanity.com/paper/2401.00908/) (2023)
- [DocPedia: Unleashing the Power of Large Multimodal Model in the Frequency Domain for Versatile Document Understanding](https://mlsanity.com/paper/2311.11810/) (2023)
- [LaTr: Layout-Aware Transformer for Scene-Text VQA](https://mlsanity.com/paper/2112.12494/) (2021)
- [ERNIE-Layout: Layout Knowledge Enhanced Pre-training for Visually-rich Document Understanding](https://mlsanity.com/paper/2210.06155/) (2022)
- [MM-LIMA: Less Is More for Alignment in Multi-Modal Datasets](https://mlsanity.com/paper/2308.12067/) (2023)
- [DocFormerv2: Local Features for Document Understanding](https://mlsanity.com/paper/2306.01733/) (2023)
- [OCR-IDL: OCR Annotations for Industry Document Library Dataset](https://mlsanity.com/paper/2202.12985/) (2022)
- [DocKylin: A Large Multimodal Model for Visual Document Understanding with Efficient Visual Slimming](https://mlsanity.com/paper/2406.19101/) (2024)
- [Towards Improving Document Understanding: An Exploration on Text-Grounding via MLLMs](https://mlsanity.com/paper/2311.13194/) (2023)
- [Deep Learning based Visually Rich Document Content Understanding: A Survey](https://mlsanity.com/paper/2408.01287/) (2024)
- [ICDAR 2021 Competition on Document VisualQuestion Answering](https://mlsanity.com/paper/2111.05547/) (2021)
- [Leopard: A Vision Language Model For Text-Rich Multi-Image Tasks](https://mlsanity.com/paper/2410.01744/) (2024)
- [Privacy-Aware Document Visual Question Answering](https://mlsanity.com/paper/2312.10108/) (2023)
- [TinyLVLM-eHub: Towards Comprehensive and Efficient Evaluation for Large Vision-Language Models](https://mlsanity.com/paper/2308.03729/) (2023)
- [RJUA-MedDQA: A Multimodal Benchmark for Medical Document Question Answering and Clinical Reasoning](https://mlsanity.com/paper/2402.14840/) (2024)
- [TRINS: Towards Multimodal Language Models that Can Read](https://mlsanity.com/paper/2406.06730/) (2024)
- [Analyzing the Efficacy of an LLM-Only Approach for Image-based Document Question Answering](https://mlsanity.com/paper/2309.14389/) (2023)
- [SIMPLOT: Enhancing Chart Question Answering by Distilling Essentials](https://mlsanity.com/paper/2405.00021/) (2024)
- [Test-Time Adaptation for Visual Document Understanding](https://mlsanity.com/paper/2206.07240/) (2022)
- [Exploring Sparse Spatial Relation in Graph Inference for Text-Based VQA](https://mlsanity.com/paper/2310.09147/) (2023)
- [TANQ: An open domain dataset of table answered questions](https://mlsanity.com/paper/2405.07765/) (2024)
- [MMR: Evaluating Reading Ability of Large Multimodal Models](https://mlsanity.com/paper/2408.14594/) (2024)
- [Large Language Models for Page Stream Segmentation](https://mlsanity.com/paper/2408.11981/) (2024)

## Abstract (arXiv)

This paper presents final results of ICDAR 2019 Scene Text Visual Question Answering competition (ST-VQA). ST-VQA introduces an important aspect that is not addressed by any Visual Question Answering system up to date, namely the incorporation of scene text to answer questions asked about an image. The competition introduces a new dataset comprising 23,038 images annotated with 31,791 question/answer pairs where the answer is always grounded on text instances present in the image. The images are taken from 7 different public computer vision datasets, covering a wide range of scenarios. The competition was structured in three tasks of increasing difficulty, that require reading the text in a scene and understanding it in the context of the scene, to correctly answer a given question. A novel evaluation metric is presented, which elegantly assesses both key capabilities expected from an optimal model: text recognition and image understanding. A detailed analysis of results from different participants is showcased, which provides insight into the current capabilities of VQA systems that can read. We firmly believe the dataset proposed in this challenge will be an important milestone to consider towards a path of more robust and general models that can exploit scene text to achieve holistic image understanding.