# Learning Permutation Invariant Representations using Memory Networks

arXiv: https://arxiv.org/abs/1911.07984 · HTML: https://mlsanity.com/paper/1911.07984/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 18 Nov 2019.

## Built on by (most cited first, 9 of 9)

- [Visual Language Pretrained Multiple Instance Zero-Shot Transfer for Histopathology Images](https://mlsanity.com/paper/2306.07831/) (2023)
- [Artificial Intelligence for Digital and Computational Pathology](https://mlsanity.com/paper/2401.06148/) (2023)
- [Learning Binary and Sparse Permutation-Invariant Representations for Fast and Memory Efficient Whole Slide Image Search](https://mlsanity.com/paper/2208.13653/) (2022)
- [Pay Attention with Focus: A Novel Learning Scheme for Classification of Whole Slide Images](https://mlsanity.com/paper/2106.06623/) (2021)
- [LILE: Look In-Depth before Looking Elsewhere -- A Dual Attention Network using Transformers for Cross-Modal Information Retrieval in Histopathology Archives](https://mlsanity.com/paper/2203.01445/) (2022)
- [A Short Survey on Set-Based Aggregation Techniques for Single-Vector WSI Representation in Digital Pathology](https://mlsanity.com/paper/2409.04615/) (2024)
- [Effective and Interpretable Information Aggregation with Capacity Networks](https://mlsanity.com/paper/2207.12013/) (2022)
- [Aggregation Schemes for Single-Vector WSI Representation Learning in Digital Pathology](https://mlsanity.com/paper/2501.17822/) (2025)
- [Investigating Permutation-Invariant Discrete Representation Learning for Spatially Aligned Images](https://mlsanity.com/paper/2604.01843/) (2026)

## Abstract (arXiv)

Many real-world tasks such as classification of digital histopathology images and 3D object detection involve learning from a set of instances. In these cases, only a group of instances or a set, collectively, contains meaningful information and therefore only the sets have labels, and not individual data instances. In this work, we present a permutation invariant neural network called Memory-based Exchangeable Model (MEM) for learning set functions. The MEM model consists of memory units that embed an input sequence to high-level features enabling the model to learn inter-dependencies among instances through a self-attention mechanism. We evaluated the learning ability of MEM on various toy datasets, point cloud classification, and classification of lung whole slide images (WSIs) into two subtypes of lung cancer---Lung Adenocarcinoma, and Lung Squamous Cell Carcinoma. We systematically extracted patches from lung WSIs downloaded from The Cancer Genome Atlas~(TCGA) dataset, the largest public repository of WSIs, achieving a competitive accuracy of 84.84% for classification of two sub-types of lung cancer. The results on other datasets are promising as well, and demonstrate the efficacy of our model.