# Fine-grained Image Classification and Retrieval by Combining Visual and Locally Pooled Textual Features

arXiv: https://arxiv.org/abs/2001.04732 · HTML: https://mlsanity.com/paper/2001.04732/

## Key facts

- Cited by 6 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 14 Jan 2020.

## Built on by (most cited first, 6 of 6)

- [Fine-Grained Image Analysis with Deep Learning: A Survey](https://mlsanity.com/paper/2111.06119/) (2021)
- [OCR-IDL: OCR Annotations for Industry Document Library Dataset](https://mlsanity.com/paper/2202.12985/) (2022)
- [Knowledge Mining with Scene Text for Fine-Grained Recognition](https://mlsanity.com/paper/2203.14215/) (2022)
- [MUST-VQA: MUltilingual Scene-text VQA](https://mlsanity.com/paper/2209.06730/) (2022)
- [Fine-Grained Scene Image Classification with Modality-Agnostic Adapter](https://mlsanity.com/paper/2407.02769/) (2024)
- [When Does Geometric View Synthesis Help Wine Label Retrieval? A Public One-Shot Benchmark Across Self-Supervised and Vision-Language Backbones](https://mlsanity.com/paper/2609.33359/) (2026)

## Abstract (arXiv)

Text contained in an image carries high-level semantics that can be exploited to achieve richer image understanding. In particular, the mere presence of text provides strong guiding content that should be employed to tackle a diversity of computer vision tasks such as image retrieval, fine-grained classification, and visual question answering. In this paper, we address the problem of fine-grained classification and image retrieval by leveraging textual information along with visual cues to comprehend the existing intrinsic relation between the two modalities. The novelty of the proposed model consists of the usage of a PHOC descriptor to construct a bag of textual words along with a Fisher Vector Encoding that captures the morphology of text. This approach provides a stronger multimodal representation for this task and as our experiments demonstrate, it achieves state-of-the-art results on two different tasks, fine-grained classification and image retrieval.