# Learning Binary and Sparse Permutation-Invariant Representations for Fast and Memory Efficient Whole Slide Image Search

arXiv: https://arxiv.org/abs/2208.13653 · HTML: https://mlsanity.com/paper/2208.13653/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 20 of its 42 references that are in the map.
- First posted on arXiv 29 Aug 2022.

## Builds on (most cited first, 20 of 20)

- [Densely Connected Convolutional Networks](https://mlsanity.com/paper/1608.06993/) (2016)
- [Deep Sets](https://mlsanity.com/paper/1703.06114/) (2017)
- [Attention-based Deep Multiple Instance Learning](https://mlsanity.com/paper/1802.04712/) (2018)
- [An Introduction to Variational Autoencoders](https://mlsanity.com/paper/1906.02691/) (2019)
- [Patch-based Convolutional Neural Network for Whole Slide Tissue Image Classification](https://mlsanity.com/paper/1504.07947/) (2015)
- [A Survey on Learning to Hash](https://mlsanity.com/paper/1606.00185/) (2016)
- [Neural Image Compression for Gigapixel Histopathology Image Analysis](https://mlsanity.com/paper/1811.02840/) (2018)
- [Deep Learning for Whole Slide Image Analysis: An Overview](https://mlsanity.com/paper/1910.11097/) (2019)
- [Similar Image Search for Histopathology: SMILY](https://mlsanity.com/paper/1901.11112/) (2019)
- [Pan-Cancer Diagnostic Consensus Through Searching Archival Histopathology Images Using Artificial Intelligence](https://mlsanity.com/paper/1911.08736/) (2019)
- [SOS: Selective Objective Switch for Rapid Immunofluorescence Whole Slide Image Classification](https://mlsanity.com/paper/2003.05080/) (2020)
- [Deep Quantization: Encoding Convolutional Activations with Deep Generative Model](https://mlsanity.com/paper/1611.09502/) (2016)
- [Reinforced Auto-Zoom Net: Towards Accurate and Fast Breast Cancer Segmentation in Whole-slide Images](https://mlsanity.com/paper/1807.11113/) (2018)
- [MinMax Radon Barcodes for Medical Image Retrieval](https://mlsanity.com/paper/1610.00318/) (2016)
- [Deep Barcodes for Fast Retrieval of Histopathology Scans](https://mlsanity.com/paper/1805.08833/) (2018)
- [Learning Permutation Invariant Representations using Memory Networks](https://mlsanity.com/paper/1911.07984/) (2019)
- [Context-aware stacked convolutional neural networks for classification of breast carcinomas in whole-slide histopathology images](https://mlsanity.com/paper/1705.03678/) (2017)
- [Adversarial Fisher Vectors for Unsupervised Representation Learning](https://mlsanity.com/paper/1910.13101/) (2019)
- [Pay Attention with Focus: A Novel Learning Scheme for Classification of Whole Slide Images](https://mlsanity.com/paper/2106.06623/) (2021)
- [A non-alternating graph hashing algorithm for large scale image search](https://mlsanity.com/paper/2012.13138/) (2020)

## Built on by (most cited first, 9 of 9)

- [Beyond the Failures: Rethinking Foundation Models in Pathology](https://mlsanity.com/paper/2510.23807/) (2025)
- [TPMIL: Trainable Prototype Enhanced Multiple Instance Learning for Whole Slide Image Classification](https://mlsanity.com/paper/2305.00696/) (2023)
- [Creating an Atlas of Normal Tissue for Pruning WSI Patching Through Anomaly Detection](https://mlsanity.com/paper/2310.03106/) (2023)
- [SPLICE -- Streamlining Digital Pathology Image Processing](https://mlsanity.com/paper/2404.17704/) (2024)
- [Selection of Distinct Morphologies to Divide & Conquer Gigapixel Pathology Images](https://mlsanity.com/paper/2311.09902/) (2023)
- [Siamese Content-based Search Engine for a More Transparent Skin and Breast Cancer Diagnosis through Histological Imaging](https://mlsanity.com/paper/2401.08272/) (2024)
- [A Short Survey on Set-Based Aggregation Techniques for Single-Vector WSI Representation in Digital Pathology](https://mlsanity.com/paper/2409.04615/) (2024)
- [Multimodal Learning for Scalable Representation of High-Dimensional Medical Data](https://mlsanity.com/paper/2409.13115/) (2024)
- [Aggregation Schemes for Single-Vector WSI Representation Learning in Digital Pathology](https://mlsanity.com/paper/2501.17822/) (2025)

## Abstract (arXiv)

Learning suitable Whole slide images (WSIs) representations for efficient retrieval systems is a non-trivial task. The WSI embeddings obtained from current methods are in Euclidean space not ideal for efficient WSI retrieval. Furthermore, most of the current methods require high GPU memory due to the simultaneous processing of multiple sets of patches. To address these challenges, we propose a novel framework for learning binary and sparse WSI representations utilizing a deep generative modelling and the Fisher Vector. We introduce new loss functions for learning sparse and binary permutation-invariant WSI representations that employ instance-based training achieving better memory efficiency. The learned WSI representations are validated on The Cancer Genomic Atlas (TCGA) and Liver-Kidney-Stomach (LKS) datasets. The proposed method outperforms Yottixel (a recent search engine for histopathology images) both in terms of retrieval accuracy and speed. Further, we achieve competitive performance against SOTA on the public benchmark LKS dataset for WSI classification.