Self-distillation Augmented Masked Autoencoders for Histopathological Image Classification
Not on the map. Why?
Abstract (arXiv)
Self-supervised learning (SSL) has drawn increasing attention in histopathological image analysis in recent years. Compared to contrastive learning which is troubled with the false negative problem, i.e., semantically similar images are selected as negative samples, masked autoencoders (MAE) building SSL from a generative paradigm is probably a more appropriate pre-training. In this paper, we introduce MAE and verify the effect of visible patches for histopathological image understanding. Moreover, a novel SD-MAE model is proposed to enable a self-distillation augmented MAE. Besides the reconstruction loss on masked image patches, SD-MAE further imposes the self-distillation loss on visible patches to enhance the representational capacity of the encoder located shallow layer. We apply SD-MAE to histopathological image classification, cell segmentation and object detection. Experiments demonstrate that SD-MAE shows highly competitive performance when compared with other SSL methods in these tasks.
Key facts. Cited by 1 paper in the map (data through 7 Oct 2026). Builds on 30 of its 48 references that are in the map. First posted on arXiv 31 Mar 2022.
Builds on
30 of 48 references.
References · 30
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
- 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
- 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
- 2019Momentum Contrast for Unsupervised Visual Representation Learningcited by 7,482
- 2021Masked Autoencoders Are Scalable Vision Learnerscited by 7,339
- 2021Emerging Properties in Self-Supervised Vision Transformerscited by 6,893
- 2020Bootstrap your own latent: A new approach to self-supervised Learningcited by 4,712
- 2021BEiT: BERT Pre-Training of Image Transformerscited by 2,360
- 2021Barlow Twins: Self-Supervised Learning via Redundancy Reductioncited by 1,818
- 2021An Empirical Study of Training Self-Supervised Vision Transformerscited by 1,610
- 2018Unsupervised Representation Learning by Predicting Image Rotationscited by 1,586
- 2018Unified Perceptual Parsing for Scene Understandingcited by 1,265
18 more
- 2021SimMIM: A Simple Framework for Masked Image Modelingcited by 1,147
- 2016Colorful Image Colorizationcited by 1,113
- 2015Unsupervised Visual Representation Learning by Context Predictioncited by 1,062
- 2021VICReg: Variance-Invariance-Covariance Regularization for Self-Supervised Learningcited by 1,013
- 2021iBOT: Image BERT Pre-Training with Online Tokenizercited by 891
- 2022Exploring Plain Vision Transformer Backbones for Object Detectioncited by 723
- 2020Self-supervised Learning: Generative or Contrastivecited by 693
- 2021Masked Feature Prediction for Self-Supervised Visual Pre-Trainingcited by 573
- 2019Be Your Own Teacher: Improve the Performance of Convolutional Neural Networks via Self Distillationcited by 491
- 2020SCAN: Learning to Classify Images without Labelscited by 351
- 2016Gland Segmentation in Colon Histology Images: The GlaS Challenge Contestcited by 203
- 2020Self supervised contrastive learning for digital histopathologycited by 146
- 2019Self-supervised Feature Learning for 3D Medical Images by Playing a Rubik's Cubecited by 76
- 2022Self Pre-training with Masked Autoencoders for Medical Image Classification and Segmentationcited by 69
- 2019Multimodal Self-Supervised Learning for Medical Image Analysiscited by 57
- 2021Incremental False Negative Detection for Contrastive Learningcited by 48
- 2020Instance-aware Self-supervised Learning for Nuclei Segmentationcited by 27
- 2021Learning Representations with Contrastive Self-Supervised Learning for Histopathology Applicationscited by 12
Built on by
Citing papers per year: 2025: 1.
1 paper.