# Self-Supervised Learning of Phenotypic Representations from Cell Images with Weak Labels

arXiv: https://arxiv.org/abs/2209.07819 · HTML: https://mlsanity.com/paper/2209.07819/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 9 of its 32 references that are in the map.
- First posted on arXiv 16 Sep 2022.

## Builds on (most cited first, 9 of 9)

- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Distilling the Knowledge in a Neural Network](https://mlsanity.com/paper/1503.02531/) (2015)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [Unsupervised Learning of Visual Features by Contrasting Cluster Assignments](https://mlsanity.com/paper/2006.09882/) (2020)
- [Masked Siamese Networks for Label-Efficient Learning](https://mlsanity.com/paper/2204.07141/) (2022)
- [Multi-task Self-Supervised Visual Learning](https://mlsanity.com/paper/1708.07860/) (2017)
- [Is it Time to Replace CNNs with Transformers for Medical Images?](https://mlsanity.com/paper/2108.09038/) (2021)
- [How Transferable Are Self-supervised Features in Medical Image Classification Tasks?](https://mlsanity.com/paper/2108.10048/) (2021)
- [Contrastive Learning of Single-Cell Phenotypic Representations for Treatment Classification](https://mlsanity.com/paper/2103.16670/) (2021)

## Built on by (most cited first, 9 of 9)

- [Masked Autoencoders for Microscopy are Scalable Learners of Cellular Biology](https://mlsanity.com/paper/2404.10242/) (2024)
- [RxRx1: A Dataset for Evaluating Experimental Batch Correction Methods](https://mlsanity.com/paper/2301.05768/) (2023)
- [Masked Autoencoders are Scalable Learners of Cellular Morphology](https://mlsanity.com/paper/2309.16064/) (2023)
- [Weakly Supervised Set-Consistency Learning Improves Morphological Profiling of Single-Cell Images](https://mlsanity.com/paper/2406.05308/) (2024)
- [Class-Guided Image-to-Image Diffusion: Cell Painting from Brightfield Images with Class Labels](https://mlsanity.com/paper/2303.08863/) (2023)
- [Morphological Profiling for Drug Discovery in the Era of Deep Learning](https://mlsanity.com/paper/2312.07899/) (2023)
- [Integrating chemical structures as treatments improves representations of microscopy images for morphological profiling](https://mlsanity.com/paper/2504.09544/) (2025)
- [CLANet: A Comprehensive Framework for Cross-Batch Cell Line Identification Using Brightfield Images](https://mlsanity.com/paper/2306.16538/) (2023)
- [Adversarial Batch Representation Augmentation for Batch Correction in High-Content Cellular Screening](https://mlsanity.com/paper/2603.05622/) (2026)

## Abstract (arXiv)

We propose WS-DINO as a novel framework to use weak label information in learning phenotypic representations from high-content fluorescent images of cells. Our model is based on a knowledge distillation approach with a vision transformer backbone (DINO), and we use this as a benchmark model for our study. Using WS-DINO, we fine-tuned with weak label information available in high-content microscopy screens (treatment and compound) and achieve state-of-the-art performance in not-same-compound mechanism of action prediction on the BBBC021 dataset (98%), and not-same-compound-and-batch performance (96%) using the compound as the weak label. Our method bypasses single cell cropping as a pre-processing step, and using self-attention maps we show that the model learns structurally meaningful phenotypic profiles.