# Multi-Modal Reasoning Graph for Scene-Text Based Fine-Grained Image Classification and Retrieval

arXiv: https://arxiv.org/abs/2009.09809 · HTML: https://mlsanity.com/paper/2009.09809/

## Key facts

- Cited by 10 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 21 Sep 2020.

## Built on by (most cited first, 10 of 10)

- [Fine-Grained Image Analysis with Deep Learning: A Survey](https://mlsanity.com/paper/2111.06119/) (2021)
- [OCR-IDL: OCR Annotations for Industry Document Library Dataset](https://mlsanity.com/paper/2202.12985/) (2022)
- [Out-of-Vocabulary Challenge Report](https://mlsanity.com/paper/2209.06717/) (2022)
- [Knowledge Mining with Scene Text for Fine-Grained Recognition](https://mlsanity.com/paper/2203.14215/) (2022)
- [Large Language Models for Page Stream Segmentation](https://mlsanity.com/paper/2408.11981/) (2024)
- [MEMEWEAVER: Inter-Meme Graph Reasoning for Sexism and Misogyny Detection](https://mlsanity.com/paper/2601.08684/) (2026)
- [Towards Multimodal Vision-Language Models Generating Non-Generic Text](https://mlsanity.com/paper/2207.04174/) (2022)
- [Fine-Grained Scene Image Classification with Modality-Agnostic Adapter](https://mlsanity.com/paper/2407.02769/) (2024)
- [TrendGen: An Outfit Recommendation and Display System](https://mlsanity.com/paper/2603.27264/) (2026)
- [When Does Geometric View Synthesis Help Wine Label Retrieval? A Public One-Shot Benchmark Across Self-Supervised and Vision-Language Backbones](https://mlsanity.com/paper/2609.33359/) (2026)

## Abstract (arXiv)

Scene text instances found in natural images carry explicit semantic information that can provide important cues to solve a wide array of computer vision problems. In this paper, we focus on leveraging multi-modal content in the form of visual and textual cues to tackle the task of fine-grained image classification and retrieval. First, we obtain the text instances from images by employing a text reading system. Then, we combine textual features with salient image regions to exploit the complementary information carried by the two sources. Specifically, we employ a Graph Convolutional Network to perform multi-modal reasoning and obtain relationship-enhanced features by learning a common semantic space between salient objects and text found in an image. By obtaining an enhanced set of visual and textual features, the proposed model greatly outperforms the previous state-of-the-art in two different tasks, fine-grained classification and image retrieval in the Con-Text and Drink Bottle datasets.