# Analysis and Validation of Image Search Engines in Histopathology

arXiv: https://arxiv.org/abs/2401.03271 · HTML: https://mlsanity.com/paper/2401.03271/

## Key facts

- Cited by 7 papers in the map (data through 14 Oct 2026).
- Builds on 22 of its 77 references that are in the map.
- First posted on arXiv 6 Jan 2024.

## Builds on (most cited first, 22 of 22)

- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [Do Deep Nets Really Need to be Deep?](https://mlsanity.com/paper/1312.6184/) (2013)
- [BRACS: A Dataset for BReAst Carcinoma Subtyping in H&E Histology Images](https://mlsanity.com/paper/2111.04740/) (2021)
- [Virchow: A Million-Slide Digital Pathology Foundation Model](https://mlsanity.com/paper/2309.07778/) (2023)
- [Deep Learning for Instance Retrieval: A Survey](https://mlsanity.com/paper/2101.11282/) (2021)
- [Deep learning-based transformation of the H&E stain into special stains](https://mlsanity.com/paper/2008.08871/) (2020)
- [Fine-Tuning and Training of DenseNet for Histopathology Image Representation Using TCGA Diagnostic Slides](https://mlsanity.com/paper/2101.07903/) (2021)
- [Similar Image Search for Histopathology: SMILY](https://mlsanity.com/paper/1901.11112/) (2019)
- [Yottixel -- An Image Search Engine for Large Archives of Histopathology Whole Slide Images](https://mlsanity.com/paper/1911.08748/) (2019)
- [Resolving challenges in deep learning-based analyses of histopathological images using explanation methods](https://mlsanity.com/paper/1908.06943/) (2019)
- [Pan-Cancer Diagnostic Consensus Through Searching Archival Histopathology Images Using Artificial Intelligence](https://mlsanity.com/paper/1911.08736/) (2019)
- [MinMax Radon Barcodes for Medical Image Retrieval](https://mlsanity.com/paper/1610.00318/) (2016)
- [Deep Barcodes for Fast Retrieval of Histopathology Scans](https://mlsanity.com/paper/1805.08833/) (2018)
- [A Comparative Study of CNN, BoVW and LBP for Classification of Histopathological Images](https://mlsanity.com/paper/1710.01249/) (2017)
- [Barcode Annotations for Medical Image Retrieval: A Preliminary Investigation](https://mlsanity.com/paper/1505.05212/) (2015)
- [Comparing LBP, HOG and Deep Features for Classification of Histopathology Images](https://mlsanity.com/paper/1805.05837/) (2018)
- [Comments on 'Fast and scalable search of whole-slide images via self-supervised deep learning'](https://mlsanity.com/paper/2304.08297/) (2023)
- [Medical Image Classification via SVM using LBP Features from Saliency-Based Folded Data](https://mlsanity.com/paper/1509.04619/) (2015)
- [Deep Features for Tissue-Fold Detection in Histopathology Images](https://mlsanity.com/paper/1903.07011/) (2019)
- [Colored Kimia Path24 Dataset: Configurations and Benchmarks with Deep Embeddings](https://mlsanity.com/paper/2102.07611/) (2021)
- [Health Information Retrieval -- State of the art report](https://mlsanity.com/paper/2205.09083/) (2022)
- [Using a bag of Words for Automatic Medical Image Annotation with a Latent Semantic](https://mlsanity.com/paper/1306.0178/) (2013)

## Built on by (most cited first, 7 of 7)

- [On Validation of Search & Retrieval of Tissue Images in Digital Pathology](https://mlsanity.com/paper/2408.01570/) (2024)
- [Multimodal Learning for Scalable Representation of High-Dimensional Medical Data](https://mlsanity.com/paper/2409.13115/) (2024)
- [Aggregation Schemes for Single-Vector WSI Representation Learning in Digital Pathology](https://mlsanity.com/paper/2501.17822/) (2025)
- [Accurate and Scalable Multimodal Pathology Retrieval via Attentive Vision-Language Alignment](https://mlsanity.com/paper/2510.23224/) (2025)
- [Validation of Whole-Slide Foundation Models for Image Retrieval in TCGA Data](https://mlsanity.com/paper/2605.00902/) (2026)
- [CRISP -- Clustering-Based Redundancy-Reduced Instance Sampling for Pathology Case Representation and Retrieval](https://mlsanity.com/paper/2605.24253/) (2026)
- [Reducing Redundancy in Whole-Slide Image Patching for Scalable Indexing and Retrieval](https://mlsanity.com/paper/2606.26157/) (2026)

## Abstract (arXiv)

Searching for similar images in archives of histology and histopathology images is a crucial task that may aid in patient matching for various purposes, ranging from triaging and diagnosis to prognosis and prediction. Whole slide images (WSIs) are highly detailed digital representations of tissue specimens mounted on glass slides. Matching WSI to WSI can serve as the critical method for patient matching. In this paper, we report extensive analysis and validation of four search methods bag of visual words (BoVW), Yottixel, SISH, RetCCL, and some of their potential variants. We analyze their algorithms and structures and assess their performance. For this evaluation, we utilized four internal datasets (1269 patients) and three public datasets (1207 patients), totaling more than 200,000 patches from 38 different classes/subtypes across five primary sites. Certain search engines, for example, BoVW, exhibit notable efficiency and speed but suffer from low accuracy. Conversely, search engines like Yottixel demonstrate efficiency and speed, providing moderately accurate results. Recent proposals, including SISH, display inefficiency and yield inconsistent outcomes, while alternatives like RetCCL prove inadequate in both accuracy and efficiency. Further research is imperative to address the dual aspects of accuracy and minimal storage requirements in histopathological image search.