# Leveraging Computational Pathology AI for Noninvasive Optical Imaging Analysis Without Retraining

arXiv: https://arxiv.org/abs/2411.11613 · HTML: https://mlsanity.com/paper/2411.11613/

## Key facts

- No citing papers in the map yet (data through 14 Oct 2026).
- Builds on 11 of its 40 references that are in the map.
- First posted on arXiv 18 Nov 2024.

## Builds on (most cited first, 11 of 11)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [GPT-4 Technical Report](https://mlsanity.com/paper/2303.08774/) (2023)
- [Segment Anything](https://mlsanity.com/paper/2304.02643/) (2023)
- [Domain Generalization: A Survey](https://mlsanity.com/paper/2103.02503/) (2021)
- [HoVer-Net: Simultaneous Segmentation and Classification of Nuclei in Multi-Tissue Histology Images](https://mlsanity.com/paper/1812.06499/) (2018)
- [Domain Adaptation for Medical Image Analysis: A Survey](https://mlsanity.com/paper/2102.09508/) (2021)
- [SAM-Med2D](https://mlsanity.com/paper/2308.16184/) (2023)
- [Generalist Vision Foundation Models for Medical Imaging: A Case Study of Segment Anything Model on Zero-Shot Medical Segmentation](https://mlsanity.com/paper/2304.12637/) (2023)
- [Deep CT to MR Synthesis using Paired and Unpaired Data](https://mlsanity.com/paper/1805.10790/) (2018)
- [Multimodal Machine Learning in Image-Based and Clinical Biomedicine: Survey and Prospects](https://mlsanity.com/paper/2311.02332/) (2023)
- [Deep Learning based Skin-layer Segmentation for Characterizing Cutaneous Wounds from Optical Coherence Tomography Images](https://mlsanity.com/paper/2306.01252/) (2023)

## Abstract (arXiv)

Noninvasive optical imaging modalities can probe patient's tissue in 3D and over time generate gigabytes of clinically relevant data per sample. There is a need for AI models to analyze this data and assist clinical workflow. The lack of expert labelers and the large dataset required (>100,000 images) for model training and tuning are the main hurdles in creating foundation models. In this paper we introduce FoundationShift, a method to apply any AI model from computational pathology without retraining. We show our method is more accurate than state of the art models (SAM, MedSAM, SAM-Med2D, CellProfiler, Hover-Net, PLIP, UNI and ChatGPT), with multiple imaging modalities (OCT and RCM). This is achieved without the need for model retraining or fine-tuning. Applying our method to noninvasive in vivo images could enable physicians to readily incorporate optical imaging modalities into their clinical practice, providing real time tissue analysis and improving patient care.