mlsanityData to 7 Oct 2026

Self-Supervised Vision Transformers Learn Visual Concepts in Histopathology

arXiv 2203.00585 ↗First posted 1 Mar 2022cs.CV q-bio.TOcited by 49

Not on the map. Why?

Abstract (arXiv)

Tissue phenotyping is a fundamental task in learning objective characterizations of histopathologic biomarkers within the tumor-immune microenvironment in cancer pathology. However, whole-slide imaging (WSI) is a complex computer vision in which: 1) WSIs have enormous image resolutions with precludes large-scale pixel-level efforts in data curation, and 2) diversity of morphological phenotypes results in inter- and intra-observer variability in tissue labeling. To address these limitations, current efforts have proposed using pretrained image encoders (transfer learning from ImageNet, self-supervised pretraining) in extracting morphological features from pathology, but have not been extensively validated. In this work, we conduct a search for good representations in pathology by training a variety of self-supervised models with validation on a variety of weakly-supervised and patch-level tasks. Our key finding is in discovering that Vision Transformers using DINO-based knowledge distillation are able to learn data-efficient and interpretable features in histology images wherein the different attention heads learn distinct morphological phenotypes. We make evaluation code and pretrained weights publicly-available at: https://github.com/Richarizardd/Self-Supervised-ViT-Path.

Key facts. Cited by 49 papers in the map (data through 7 Oct 2026). Builds on 28 of its 46 references that are in the map. First posted on arXiv 1 Mar 2022.

Builds on

28 of 46 references.

References · 28

  1. 2017Attention Is All You Needcited by 59,932
  2. 2015Deep Residual Learning for Image Recognitioncited by 51,809
  3. 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
  4. 2018Representation Learning with Contrastive Predictive Codingcited by 7,662
  5. 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
  6. 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
  7. 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,712
  8. 2020Exploring Simple Siamese Representation Learningcited by 2,853
  9. 2021MLP-Mixer: An all-MLP Architecture for Visioncited by 1,441
  10. 2017Deep Setscited by 1,394
  11. 2018Attention-based Deep Multiple Instance Learningcited by 1,077
  12. 2020Data Efficient and Weakly Supervised Computational Pathology on Whole Slide Imagescited by 738
16 more
  1. 2020Dual-stream Multiple Instance Learning Network for Whole Slide Image Classification with Self-supervised Contrastive Learningcited by 473
  2. 2020Debiased Contrastive Learningcited by 354
  3. 2019Large Scale Adversarial Representation Learningcited by 238
  4. 2021Localizing Objects with Self-Supervised Transformers and no Labelscited by 227
  5. 2019Pathomic Fusion: An Integrated Framework for Fusing Histopathology and Genomic Features for Cancer Diagnosis and Prognosiscited by 165
  6. 2020Self supervised contrastive learning for digital histopathologycited by 146
  7. 2021Whole Slide Images are 2D Point Clouds: Context-Aware Survival Prediction using Patch-based Graph Convolutional Networkscited by 124
  8. 2016Towards a Neural Statisticiancited by 119
  9. 2020Self-Path: Self-supervision for Classification of Pathology Images with Limited Annotationscited by 89
  10. 2020Self-Supervision Closes the Gap Between Weak and Strong Supervision in Histologycited by 43
  11. 2021Self-supervised driven consistency training for annotation efficient histopathology image analysiscited by 39
  12. 2019Semi-Supervised Histology Classification using Deep Multiple Instance Learning and Contrastive Predictive Codingcited by 35
  13. 2021Self supervised learning improves dMMR/MSI detection from histology slides across multiple cancerscited by 23
  14. 2019Weakly Supervised Prostate TMA Classification via Graph Convolutional Networkscited by 17
  15. 2021Adversarial learning of cancer tissue representationscited by 8
  16. 2021Resource and data efficient self supervised learningcited by 4

Built on by

Cited by year (2026 to 7 Oct)

Citing papers per year: 2022: 11; 2023: 14; 2024: 15; 2025: 7; 2026: 2.

49 papers.

Sort:

By area:

  1. 2022Scaling Vision Transformers to Gigapixel Images via Hierarchical Self-Supervised Learningcited by 357
  2. 2023Artificial Intelligence for Digital and Computational Pathologycited by 106
  3. 2024Phikon-v2, A large and public feature extractor for biomarker predictioncited by 67
  4. 2023Task-specific Fine-tuning via Variational Information Bottleneck for Weakly-supervised Pathology Whole Slide Image Classificationcited by 54
  5. 2023A General-Purpose Self-Supervised Model for Computational Pathologycited by 42
  6. 2023The Rise of AI Language Pathologists: Exploring Two-level Prompt Learning for Few-shot Weakly-supervised Whole Slide Image Classificationcited by 41
  7. 2024RudolfV: A Foundation Model by Pathologists for Pathologistscited by 40
  8. 2022Towards Label-efficient Automatic Diagnosis and Analysis: A Comprehensive Survey of Advanced Deep Learning-based Weakly-supervised, Semi-supervised and Self-supervised Techniques in Histopathological Image Analysiscited by 29
  9. 2022Multiple Instance Learning via Iterative Self-Paced Supervised Contrastive Learningcited by 22
  10. 2023Rethinking Multiple Instance Learning for Whole Slide Image Classification: A Good Instance Classifier is All You Needcited by 21
  11. 2023Limited Data, Unlimited Potential: A Study on ViTs Augmented by Masked Autoencoderscited by 17
  12. 2023Domain-specific optimization and diverse evaluation of self-supervised models for histopathologycited by 11
37 more
  1. 2022From Modern CNNs to Vision Transformers: Assessing the Performance, Robustness, and Classification Strategies of Deep Learning Models in Histopathologycited by 9
  2. 2022Vision Transformers in Medical Imaging: A Reviewcited by 8
  3. 2024AI-based Anomaly Detection for Clinical-Grade Histopathological Diagnosticscited by 8
  4. 2024HoloHisto: End-to-end Gigapixel WSI Segmentation with 4K Resolution Sequential Tokenizationcited by 8
  5. 2025Self-Supervision Enhances Instance-based Multiple Instance Learning Methods in Digital Pathology: A Benchmark Studycited by 6
  6. 2023OpenAL: An Efficient Deep Active Learning Framework for Open-Set Pathology Image Classificationcited by 5
  7. 2024Lifelong Histopathology Whole Slide Image Retrieval via Distance Consistency Rehearsalcited by 5
  8. 2025Towards Effective and Efficient Context-aware Nucleus Detection in Histopathology Whole Slide Imagescited by 5
  9. 2023Attention De-sparsification Matters: Inducing Diversity in Digital Pathology Representation Learningcited by 4
  10. 2024Hierarchical Text-to-Vision Self Supervised Alignment for Improved Histopathology Representation Learningcited by 4
  11. 2025Advanced Deep Learning and Large Language Models: Comprehensive Insights for Cancer Detectioncited by 4
  12. 2022Self-supervised contrastive learning of echocardiogram videos enables label-efficient cardiac disease diagnosiscited by 3
  13. 2022Multi-modal Masked Autoencoders Learn Compositional Histopathological Representationscited by 3
  14. 2023Out-Of-Domain Unlabeled Data Improves Generalizationcited by 3
  15. 2024BioFusionNet: Deep Learning-Based Survival Risk Stratification in ER+ Breast Cancer Through Multifeature and Multimodal Data Fusioncited by 3
  16. 2022Patient-level Microsatellite Stability Assessment from Whole Slide Images By Combining Momentum Contrast Learning and Group Patch Embeddingscited by 2
  17. 2022HistoPerm: A Permutation-Based View Generation Approach for Improving Histopathologic Feature Representation Learningcited by 2
  18. 2024Genomics-guided Representation Learning for Pathologic Pan-cancer Tumor Microenvironment Subtype Predictioncited by 2
  19. 2025Benchmarking Foundation Models for Mitotic Figure Classificationcited by 2
  20. 2025Dynamic Residual Encoding with Slide-Level Contrastive Learning for End-to-End Whole Slide Image Representationcited by 2
  21. 2022Language models are good pathologists: using attention-based sequence reduction and text-pretrained transformers for efficient WSI classificationcited by 1
  22. 2022Seeded iterative clustering for histology region identificationcited by 1
  23. 2023A vision transformer-based framework for knowledge transfer from multi-modal to mono-modal lymphoma subtyping modelscited by 1
  24. 2024Primary liver cancer classification from routine tumour biopsy using weakly supervised deep learningcited by 1
  25. 2024Deep Learning-based Prediction of Breast Cancer Tumor and Immune Phenotypes from Histopathologycited by 1
  26. 2024Predicting Genetic Mutation from Whole Slide Images via Biomedical-Linguistic Knowledge Enhanced Multi-label Classificationcited by 1
  27. 2024Rethinking Pre-Trained Feature Extractor Selection in Multiple Instance Learning for Whole Slide Image Classificationcited by 1
  28. 2023Classification in Histopathology: A unique deep embeddings extractor for multiple classification taskscited by 0
  29. 2023LESS: Label-efficient Multi-scale Learning for Cytological Whole Slide Image Screeningcited by 0
  30. 2023Semi-supervised ViT knowledge distillation network with style transfer normalization for colorectal liver metastases survival predictioncited by 0
  31. 2024Reducing self-supervised learning complexity improves weakly-supervised classification performance in computational pathologycited by 0
  32. 2024Benchmarking Hierarchical Image Pyramid Transformer for the classification of colon biopsies and polyps in histopathology imagescited by 0
  33. 2024PreMix: Label-Efficient Multiple Instance Learning via Non-Contrastive Pre-training and Feature Mixingcited by 0
  34. 2025Hierarchical Self-Supervised Adversarial Training for Robust Vision Models in Histopathologycited by 0
  35. 2025MedGEN-Bench: A Contextually Entangled Benchmark for Open-ended Multimodal Medical Generationcited by 0
  36. 2026Consistency-Driven Co-Evolution for Self-Supervised Cross-Representation Learningcited by 0
  37. 2026Learning Where to Focus: Self-Supervised Multi-Scale ViTs for Histopathologycited by 0