# Evaluating Cell AI Foundation Models in Kidney Pathology with Human-in-the-Loop Enrichment

arXiv: https://arxiv.org/abs/2411.00078 · HTML: https://mlsanity.com/paper/2411.00078/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- Builds on 18 of its 53 references that are in the map.
- First posted on arXiv 31 Oct 2024.

## Builds on (most cited first, 18 of 18)

- [Mask R-CNN](https://mlsanity.com/paper/1703.06870/) (2017)
- [On the Opportunities and Risks of Foundation Models](https://mlsanity.com/paper/2108.07258/) (2021)
- [A Survey on Deep Learning in Medical Image Analysis](https://mlsanity.com/paper/1702.05747/) (2017)
- [Medical SAM Adapter: Adapting Segment Anything Model for Medical Image Segmentation](https://mlsanity.com/paper/2304.12620/) (2023)
- [HoVer-Net: Simultaneous Segmentation and Classification of Nuclei in Multi-Tissue Histology Images](https://mlsanity.com/paper/1812.06499/) (2018)
- [Scaling Vision Transformers to Gigapixel Images via Hierarchical Self-Supervised Learning](https://mlsanity.com/paper/2206.02647/) (2022)
- [Segment Anything Model for Medical Image Analysis: an Experimental Study](https://mlsanity.com/paper/2304.10517/) (2023)
- [Segment Anything in Medical Images](https://mlsanity.com/paper/2304.12306/) (2023)
- [Segment Anything Model (SAM) for Digital Pathology: Assess Zero-shot Segmentation on Whole Slide Imaging](https://mlsanity.com/paper/2304.04155/) (2023)
- [CellViT: Vision Transformers for Precise Cell Segmentation and Classification](https://mlsanity.com/paper/2306.15350/) (2023)
- [Lizard: A Large-Scale Dataset for Colonic Nuclear Instance Segmentation and Classification](https://mlsanity.com/paper/2108.11195/) (2021)
- [VISTA3D: A Unified Segmentation Foundation Model For 3D Medical Imaging](https://mlsanity.com/paper/2406.05285/) (2024)
- [Micro-Net: A unified model for segmentation of various objects in microscopy images](https://mlsanity.com/paper/1804.08145/) (2018)
- [CPP-Net: Context-aware Polygon Proposal Network for Nucleus Segmentation](https://mlsanity.com/paper/2102.06867/) (2021)
- [Nuclei instance segmentation and classification in histopathology images with StarDist](https://mlsanity.com/paper/2203.02284/) (2022)
- [Foundation Models for Biomedical Image Segmentation: A Survey](https://mlsanity.com/paper/2401.07654/) (2024)
- [HATs: Hierarchical Adaptive Taxonomy Segmentation for Panoramic Pathology Image Analysis](https://mlsanity.com/paper/2407.00596/) (2024)
- [Nuclei & Glands Instance Segmentation in Histology Images: A Narrative Review](https://mlsanity.com/paper/2208.12460/) (2022)

## Built on by (most cited first, 5 of 5)

- [DeepAndes: A Self-Supervised Vision Foundation Model for Multi-Spectral Remote Sensing Imagery of the Andes](https://mlsanity.com/paper/2504.20303/) (2025)
- [AMA-SAM: Adversarial Multi-Domain Alignment of Segment Anything Model for High-Fidelity Histology Nuclei Segmentation](https://mlsanity.com/paper/2503.21695/) (2025)
- [ZeroReg3D: A Zero-shot Registration Pipeline for 3D Consecutive Histopathology Image Reconstruction](https://mlsanity.com/paper/2506.21923/) (2025)
- [Explainable Pathomics Feature Visualization via Correlation-aware Conditional Feature Editing](https://mlsanity.com/paper/2602.05397/) (2026)
- [Scalable In-Domain Self-Supervised Foundation Model for Dense Representation Transfer in High-Resolution Plant Imaging](https://mlsanity.com/paper/2609.32183/) (2026)

## Abstract (arXiv)

Training AI foundation models has emerged as a promising large-scale learning approach for addressing real-world healthcare challenges, including digital pathology. While many of these models have been developed for tasks like disease diagnosis and tissue quantification using extensive and diverse training datasets, their readiness for deployment on some arguably simplest tasks, such as nuclei segmentation within a single organ (e.g., the kidney), remains uncertain. This paper seeks to answer this key question, "How good are we?", by thoroughly evaluating the performance of recent cell foundation models on a curated multi-center, multi-disease, and multi-species external testing dataset. Additionally, we tackle a more challenging question, "How can we improve?", by developing and assessing human-in-the-loop data enrichment strategies aimed at enhancing model performance while minimizing the reliance on pixel-level human annotation. To address the first question, we curated a multicenter, multidisease, and multispecies dataset consisting of 2,542 kidney whole slide images (WSIs). Three state-of-the-art (SOTA) cell foundation models-Cellpose, StarDist, and CellViT-were selected for evaluation. To tackle the second question, we explored data enrichment algorithms by distilling predictions from the different foundation models with a human-in-the-loop framework, aiming to further enhance foundation model performance with minimal human efforts. Our experimental results showed that all three foundation models improved over their baselines with model fine-tuning with enriched data. Interestingly, the baseline model with the highest F1 score does not yield the best segmentation outcomes after fine-tuning. This study establishes a benchmark for the development and deployment of cell vision foundation models tailored for real-world data applications.