# Deep Learning for BioImaging: What Are We Really Learning?

arXiv: https://arxiv.org/abs/2603.13377 · HTML: https://mlsanity.com/paper/2603.13377/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 41 of its 78 references that are in the map.
- First posted on arXiv 10 Mar 2026.

## Builds on (most cited first, 30 of 41)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://mlsanity.com/paper/2103.14030/) (2021)
- [Semi-Supervised Classification with Graph Convolutional Networks](https://mlsanity.com/paper/1609.02907/) (2016)
- [ImageNet Large Scale Visual Recognition Challenge](https://mlsanity.com/paper/1409.0575/) (2014)
- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [Emerging Properties in Self-Supervised Vision Transformers](https://mlsanity.com/paper/2104.14294/) (2021)
- [XGBoost: A Scalable Tree Boosting System](https://mlsanity.com/paper/1603.02754/) (2016)
- [Neural Message Passing for Quantum Chemistry](https://mlsanity.com/paper/1704.01212/) (2017)
- [WILDS: A Benchmark of in-the-Wild Distribution Shifts](https://mlsanity.com/paper/2012.07421/) (2020)
- [Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gauges](https://mlsanity.com/paper/2104.13478/) (2021)
- [Deep Image Prior](https://mlsanity.com/paper/1711.10925/) (2017)
- [Attention-based Deep Multiple Instance Learning](https://mlsanity.com/paper/1802.04712/) (2018)
- [iBOT: Image BERT Pre-Training with Online Tokenizer](https://mlsanity.com/paper/2111.07832/) (2021)
- [Do Vision Transformers See Like Convolutional Neural Networks?](https://mlsanity.com/paper/2108.08810/) (2021)
- [ImageNet-21K Pretraining for the Masses](https://mlsanity.com/paper/2104.10972/) (2021)
- [What's Hidden in a Randomly Weighted Neural Network?](https://mlsanity.com/paper/1911.13299/) (2019)
- [Benchmarking Self-Supervised Learning on Diverse Pathology Datasets](https://mlsanity.com/paper/2212.04690/) (2022)
- [BRACS: A Dataset for BReAst Carcinoma Subtyping in H&E Histology Images](https://mlsanity.com/paper/2111.04740/) (2021)
- [Deep Decoder: Concise Image Representations from Untrained Non-convolutional Networks](https://mlsanity.com/paper/1810.03982/) (2018)
- [HEST-1k: A Dataset for Spatial Transcriptomics and Histology Image Analysis](https://mlsanity.com/paper/2406.16192/) (2024)
- [Phikon-v2, A large and public feature extractor for biomarker prediction](https://mlsanity.com/paper/2409.09173/) (2024)
- [Hierarchical Graph Representations in Digital Pathology](https://mlsanity.com/paper/2102.11057/) (2021)
- [A Multimodal Knowledge-enhanced Whole-slide Pathology Foundation Model](https://mlsanity.com/paper/2407.15362/) (2024)
- [Hibou: A Family of Foundational Vision Transformers for Pathology](https://mlsanity.com/paper/2406.05074/) (2024)
- [Masked Autoencoders for Microscopy are Scalable Learners of Cellular Biology](https://mlsanity.com/paper/2404.10242/) (2024)
- [RxRx1: A Dataset for Evaluating Experimental Batch Correction Methods](https://mlsanity.com/paper/2301.05768/) (2023)
- [Revisiting the Platonic Representation Hypothesis: An Aristotelian View](https://mlsanity.com/paper/2602.14486/) (2026)
- [Vision Transformers Don't Need Trained Registers](https://mlsanity.com/paper/2506.08010/) (2025)
- [Inherently Interpretable Time Series Classification via Multiple Instance Learning](https://mlsanity.com/paper/2311.10049/) (2023)

## Abstract (arXiv)

Representation learning has driven major advances in natural image analysis by enabling models to acquire high-level semantic features. In microscopy imaging, however, it remains unclear what current representation learning methods really learn. In this work, we conduct a systematic study of representation learning for the two most widely used and broadly available microscopy data types, representing critical scales in biology: cell culture and tissue imaging. We investigate whether, in contrast to natural images, existing models fail to consistently acquire high-level, biologically meaningful features. To this end, we introduce a set of simple yet revealing baselines on curated benchmarks, including untrained models and structural representations of cellular tissue. Our results show that, surprisingly, for a considerable subset of evaluation settings, the baselines are comparable to state-of-the-art methods, demonstrating that many commonly used benchmark metrics are insufficient to assess representation quality and often mask a lack of relevant high-level abstractions. In addition, we investigate how detailed comparisons with these baselines provide ways to interpret the strengths and weaknesses of models for further improvements. Together, our results suggest that progress in representation learning for microscopy requires not only stronger models, but also benchmarks that are more indicative of what is actually learned.