# Siamese Content-based Search Engine for a More Transparent Skin and Breast Cancer Diagnosis through Histological Imaging

arXiv: https://arxiv.org/abs/2401.08272 · HTML: https://mlsanity.com/paper/2401.08272/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 12 of its 72 references that are in the map.
- First posted on arXiv 16 Jan 2024.

## Builds on (most cited first, 12 of 12)

- [Similar Image Search for Histopathology: SMILY](https://mlsanity.com/paper/1901.11112/) (2019)
- [Yottixel -- An Image Search Engine for Large Archives of Histopathology Whole Slide Images](https://mlsanity.com/paper/1911.08748/) (2019)
- [A Survey on Graph-Based Deep Learning for Computational Histopathology](https://mlsanity.com/paper/2107.00272/) (2021)
- [Medical Image Retrieval using Deep Convolutional Neural Network](https://mlsanity.com/paper/1703.08472/) (2017)
- [Fisher Discriminant Triplet and Contrastive Losses for Training Siamese Networks](https://mlsanity.com/paper/2004.04674/) (2020)
- [Vision Transformers for Small Histological Datasets Learned through Knowledge Distillation](https://mlsanity.com/paper/2305.17370/) (2023)
- [Learning Binary and Sparse Permutation-Invariant Representations for Fast and Memory Efficient Whole Slide Image Search](https://mlsanity.com/paper/2208.13653/) (2022)
- [Towards More Transparent and Accurate Cancer Diagnosis with an Unsupervised CAE Approach](https://mlsanity.com/paper/2305.11728/) (2023)
- [Self-supervised learning of a tailored Convolutional Auto Encoder for histopathological prostate grading](https://mlsanity.com/paper/2303.11837/) (2023)
- [WWFedCBMIR: World-Wide Federated Content-Based Medical Image Retrieval](https://mlsanity.com/paper/2305.03383/) (2023)
- [HistoKT: Cross Knowledge Transfer in Computational Pathology](https://mlsanity.com/paper/2201.11246/) (2022)
- [Joint Triplet Autoencoder for Histopathological Colon Cancer Nuclei Retrieval](https://mlsanity.com/paper/2105.10262/) (2021)

## Built on by (most cited first, 2 of 2)

- [THIR: Topological Histopathological Image Retrieval](https://mlsanity.com/paper/2511.13170/) (2025)
- [Predicting the risk of colorectal anastomotic leak based on preoperative mapping of the blood supply of the bowel](https://mlsanity.com/paper/2606.02156/) (2026)

## Abstract (arXiv)

Computer Aid Diagnosis (CAD) has developed digital pathology with Deep Learning (DL)-based tools to assist pathologists in decision-making. Content-Based Histopathological Image Retrieval (CBHIR) is a novel tool to seek highly correlated patches in terms of similarity in histopathological features. In this work, we proposed two CBHIR approaches on breast (Breast-twins) and skin cancer (Skin-twins) data sets for robust and accurate patch-level retrieval, integrating a custom-built Siamese network as a feature extractor. The proposed Siamese network is able to generalize for unseen images by focusing on the similar histopathological features of the input pairs. The proposed CBHIR approaches are evaluated on the Breast (public) and Skin (private) data sets with top K accuracy. Finding the optimum amount of K is challenging, but also, as much as K increases, the dissimilarity between the query and the returned images increases which might mislead the pathologists. To the best of the author's belief, this paper is tackling this issue for the first time on histopathological images by evaluating the top first retrieved images. The Breast-twins model achieves 70% of the F1score at the top first, which exceeds the other state-of-the-art methods at a higher amount of K such as 5 and 400. Skin-twins overpasses the recently proposed Convolutional Auto Encoder (CAE) by 67%, increasing the precision. Besides, the Skin-twins model tackles the challenges of Spitzoid Tumors of Uncertain Malignant Potential (STUMP) to assist pathologists with retrieving top K images and their corresponding labels. So, this approach can offer a more explainable CAD tool to pathologists in terms of transparency, trustworthiness, or reliability among other characteristics.