# PLUTO: Pathology-Universal Transformer

arXiv: https://arxiv.org/abs/2405.07905 · HTML: https://mlsanity.com/paper/2405.07905/

## Key facts

- Cited by 17 papers in the map (data through 14 Oct 2026).
- Builds on 30 of its 57 references that are in the map.
- First posted on arXiv 13 May 2024.

## Builds on (most cited first, 30 of 30)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Masked Autoencoders Are Scalable Vision Learners](https://mlsanity.com/paper/2111.06377/) (2021)
- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [Scaling Laws for Neural Language Models](https://mlsanity.com/paper/2001.08361/) (2020)
- [On the Opportunities and Risks of Foundation Models](https://mlsanity.com/paper/2108.07258/) (2021)
- [WILDS: A Benchmark of in-the-Wild Distribution Shifts](https://mlsanity.com/paper/2012.07421/) (2020)
- [Per-Pixel Classification is Not All You Need for Semantic Segmentation](https://mlsanity.com/paper/2107.06278/) (2021)
- [ShuffleNet V2: Practical Guidelines for Efficient CNN Architecture Design](https://mlsanity.com/paper/1807.11164/) (2018)
- [Attention-based Deep Multiple Instance Learning](https://mlsanity.com/paper/1802.04712/) (2018)
- [iBOT: Image BERT Pre-Training with Online Tokenizer](https://mlsanity.com/paper/2111.07832/) (2021)
- [Vision Transformer Adapter for Dense Predictions](https://mlsanity.com/paper/2205.08534/) (2022)
- [HoVer-Net: Simultaneous Segmentation and Classification of Nuclei in Multi-Tissue Histology Images](https://mlsanity.com/paper/1812.06499/) (2018)
- [Scaling Vision Transformers to Gigapixel Images via Hierarchical Self-Supervised Learning](https://mlsanity.com/paper/2206.02647/) (2022)
- [Context Autoencoder for Self-Supervised Representation Learning](https://mlsanity.com/paper/2202.03026/) (2022)
- [Gland Segmentation in Colon Histology Images: The GlaS Challenge Contest](https://mlsanity.com/paper/1603.00275/) (2016)
- [Quantifying the effects of data augmentation and stain color normalization in convolutional neural networks for computational pathology](https://mlsanity.com/paper/1902.06543/) (2019)
- [FlexiViT: One Model for All Patch Sizes](https://mlsanity.com/paper/2212.08013/) (2022)
- [Benchmarking Self-Supervised Learning on Diverse Pathology Datasets](https://mlsanity.com/paper/2212.04690/) (2022)
- [Deep Learning for Identifying Metastatic Breast Cancer](https://mlsanity.com/paper/1606.05718/) (2016)
- [Automated Gleason Grading of Prostate Biopsies using Deep Learning](https://mlsanity.com/paper/1907.07980/) (2019)
- [Additive MIL: Intrinsically Interpretable Multiple Instance Learning for Pathology](https://mlsanity.com/paper/2206.01794/) (2022)
- [Computational Pathology: A Survey Review and The Way Forward](https://mlsanity.com/paper/2304.05482/) (2023)
- [Unleashing the Power of Prompt-driven Nucleus Instance Segmentation](https://mlsanity.com/paper/2311.15939/) (2023)
- [A Novel Benchmark for Few-Shot Semantic Segmentation in the Era of Foundation Models](https://mlsanity.com/paper/2401.11311/) (2024)
- [FreMIM: Fourier Transform Meets Masked Image Modeling for Medical Image Segmentation](https://mlsanity.com/paper/2304.10864/) (2023)
- [ContriMix: Scalable stain color augmentation for domain generalization without domain labels in digital pathology](https://mlsanity.com/paper/2306.04527/) (2023)
- [Synthetic DOmain-Targeted Augmentation (S-DOTA) Improves Model Generalization in Digital Pathology](https://mlsanity.com/paper/2305.02401/) (2023)
- [Self-training of Machine Learning Models for Liver Histopathology: Generalization under Clinical Shifts](https://mlsanity.com/paper/2211.07692/) (2022)
- [Scale Stain: Multi-Resolution Feature Enhancement in Pathology Visualization](https://mlsanity.com/paper/1610.04141/) (2016)

## Built on by (most cited first, 17 of 17)

- [Multimodal Whole Slide Foundation Model for Pathology](https://mlsanity.com/paper/2411.19666/) (2024)
- [Virchow2: Scaling Self-Supervised Mixed Magnification Models in Pathology](https://mlsanity.com/paper/2408.00738/) (2024)
- [A Survey of Pathology Foundation Model: Progress and Future Directions](https://mlsanity.com/paper/2504.04045/) (2025)
- [Atlas: A Novel Pathology Foundation Model by Mayo Clinic, Charit'e, and Aignostics](https://mlsanity.com/paper/2501.05409/) (2025)
- [A New Era in Computational Pathology: A Survey on Foundation and Vision-Language Models](https://mlsanity.com/paper/2408.14496/) (2024)
- [SurGen: 1020 H&E-stained Whole Slide Images With Survival and Genetic Markers](https://mlsanity.com/paper/2502.04946/) (2025)
- [Foundation Models in Medical Imaging: A Review and Outlook](https://mlsanity.com/paper/2506.09095/) (2025)
- [Atlas 2 -- Foundation models for clinical deployment](https://mlsanity.com/paper/2601.05148/) (2026)
- [From Classical Machine Learning to Emerging Foundation Models: Review on Multimodal Data Integration for Cancer Research](https://mlsanity.com/paper/2507.09028/) (2025)
- [Multimodal Model for Computational Pathology:Representation Learning and Image Compression](https://mlsanity.com/paper/2603.18660/) (2026)
- [PLUTO-4: Frontier Pathology Foundation Models](https://mlsanity.com/paper/2511.02826/) (2025)
- [Foundation Models in Biomedical Imaging: Turning Hype into Reality](https://mlsanity.com/paper/2512.15808/) (2025)
- [Mixed Magnification Aggregation for Generalizable Region-Level Representations in Computational Pathology](https://mlsanity.com/paper/2602.22176/) (2026)
- [A Comprehensive Benchmark of Histopathology Foundation Models for Kidney Digital Pathology Images](https://mlsanity.com/paper/2603.15967/) (2026)
- [Advancing Cross-Organ Domain Generalization with Test-Time Style Transfer and Diversity Enhancement](https://mlsanity.com/paper/2503.18567/) (2025)
- [Out-of-distribution evaluations of channel agnostic masked autoencoders in fluorescence microscopy](https://mlsanity.com/paper/2503.19149/) (2025)
- [HERO: Histology Encoder for Robust Representation in Oncology](https://mlsanity.com/paper/2609.35943/) (2026)

## Abstract (arXiv)

Pathology is the study of microscopic inspection of tissue, and a pathology diagnosis is often the medical gold standard to diagnose disease. Pathology images provide a unique challenge for computer-vision-based analysis: a single pathology Whole Slide Image (WSI) is gigapixel-sized and often contains hundreds of thousands to millions of objects of interest across multiple resolutions. In this work, we propose PathoLogy Universal TransfOrmer (PLUTO): a light-weight pathology FM that is pre-trained on a diverse dataset of 195 million image tiles collected from multiple sites and extracts meaningful representations across multiple WSI scales that enable a large variety of downstream pathology tasks. In particular, we design task-specific adaptation heads that utilize PLUTO's output embeddings for tasks which span pathology scales ranging from subcellular to slide-scale, including instance segmentation, tile classification, and slide-level prediction. We compare PLUTO's performance to other state-of-the-art methods on a diverse set of external and internal benchmarks covering multiple biologically relevant tasks, tissue types, resolutions, stains, and scanners. We find that PLUTO matches or outperforms existing task-specific baselines and pathology-specific foundation models, some of which use orders-of-magnitude larger datasets and model sizes when compared to PLUTO. Our findings present a path towards a universal embedding to power pathology image analysis, and motivate further exploration around pathology foundation models in terms of data diversity, architectural improvements, sample efficiency, and practical deployability in real-world applications.