# Are Vision Foundation Models Foundational for Electron Microscopy Image Segmentation?

arXiv: https://arxiv.org/abs/2602.08505 · HTML: https://mlsanity.com/paper/2602.08505/

## Key facts

- Cited by 1 papers in the map (data through 14 Oct 2026).
- Builds on 23 of its 47 references that are in the map.
- First posted on arXiv 9 Feb 2026.

## Builds on (most cited first, 23 of 23)

- [U-Net: Convolutional Networks for Biomedical Image Segmentation](https://mlsanity.com/paper/1505.04597/) (2015)
- [DINOv2: Learning Robust Visual Features without Supervision](https://mlsanity.com/paper/2304.07193/) (2023)
- [QLoRA: Efficient Finetuning of Quantized LLMs](https://mlsanity.com/paper/2305.14314/) (2023)
- [SAM 2: Segment Anything in Images and Videos](https://mlsanity.com/paper/2408.00714/) (2024)
- [DINOv3](https://mlsanity.com/paper/2508.10104/) (2025)
- [Reproducible scaling laws for contrastive language-image learning](https://mlsanity.com/paper/2212.07143/) (2022)
- [MONAI: An open-source framework for deep learning in healthcare](https://mlsanity.com/paper/2211.02701/) (2022)
- [nnU-Net: Self-adapting Framework for U-Net-Based Medical Image Segmentation](https://mlsanity.com/paper/1809.10486/) (2018)
- [Exposing flaws of generative model evaluation metrics and their unfair treatment of diffusion models](https://mlsanity.com/paper/2306.04675/) (2023)
- [Cell Detection with Star-convex Polygons](https://mlsanity.com/paper/1806.03535/) (2018)
- [Multimodal Whole Slide Foundation Model for Pathology](https://mlsanity.com/paper/2411.19666/) (2024)
- [BiomedParse: a biomedical foundation model for image parsing of everything everywhere all at once](https://mlsanity.com/paper/2405.12971/) (2024)
- [MTLoRA: A Low-Rank Adaptation Approach for Efficient Multi-Task Learning](https://mlsanity.com/paper/2403.20320/) (2024)
- [Does DINOv3 Set a New Medical Vision Standard? Benchmarking 2D and 3D Classification, Segmentation, and Registration](https://mlsanity.com/paper/2509.06467/) (2025)
- [Stable deep neural network architectures for mitochondria segmentation on electron microscopy volumes](https://mlsanity.com/paper/2104.03577/) (2021)
- [Self-Supervised Learning with Generative Adversarial Networks for Electron Microscopy](https://mlsanity.com/paper/2402.18286/) (2024)
- [How Foundational are Foundation Models for Time Series Forecasting?](https://mlsanity.com/paper/2510.00742/) (2025)
- [SAMDA: Leveraging SAM on Few-Shot Domain Adaptation for Electronic Microscopy Segmentation](https://mlsanity.com/paper/2403.07951/) (2024)
- [MoPEFT: A Mixture-of-PEFTs for the Segment Anything Model](https://mlsanity.com/paper/2405.00293/) (2024)
- [MIP: CLIP-based Image Reconstruction from PEFT Gradients](https://mlsanity.com/paper/2403.07901/) (2024)
- [Efficient Fine-Tuning of DINOv3 Pretrained on Natural Images for Atypical Mitotic Figure Classification](https://mlsanity.com/paper/2508.21041/) (2025)
- [CytoDINO: Risk-Aware and Biologically-Informed Adaptation of DINOv3 for Bone Marrow Cytomorphology](https://mlsanity.com/paper/2512.17930/) (2025)
- [Parameter Efficient Fine-Tuning of Segment Anything Model for Biomedical Imaging](https://mlsanity.com/paper/2502.00418/) (2025)

## Built on by (most cited first, 1 of 1)

- [Adapting Open-Weight MLLMs to Generate Point Prompts for Electron Microscopy Segmentation](https://mlsanity.com/paper/2609.14080/) (2026)

## Abstract (arXiv)

Although vision foundation models (VFMs) are increasingly reused for biomedical image analysis, it remains unclear whether the latent representations they provide are general enough to support effective transfer and reuse across heterogeneous microscopy image datasets. Here, we study this question for the problem of mitochondria segmentation in electron microscopy (EM) images, using two popular public EM datasets (Lucchi++ and VNC) and three recent representative VFMs (DINOv2, DINOv3, and OpenCLIP). We evaluate two practical model adaptation regimes: a frozen-backbone setting in which only a lightweight segmentation head is trained on top of the VFM, and parameter-efficient fine-tuning (PEFT) via Low-Rank Adaptation (LoRA) in which the VFM is fine-tuned in a targeted manner to a specific dataset. Across all backbones, we observe that training on a single EM dataset yields good segmentation performance (quantified as foreground Intersection-over-Union), and that LoRA consistently improves in-domain performance. In contrast, training on multiple EM datasets leads to severe performance degradation for all models considered, with only marginal gains from PEFT. Exploration of the latent representation space through various techniques (PCA, Fr'echet Dinov2 distance, and linear probes) reveals a pronounced and persistent domain mismatch between the two considered EM datasets in spite of their visual similarity, which is consistent with the observed failure of paired training. These results suggest that, while VFMs can deliver competitive results for EM segmentation within a single domain under lightweight adaptation, current PEFT strategies are insufficient to obtain a single robust model across heterogeneous EM datasets without additional domain-alignment mechanisms.