# Rethinking Foundation Models for Medical Image Classification through a Benchmark Study on MedMNIST

arXiv: https://arxiv.org/abs/2501.14685 · HTML: https://mlsanity.com/paper/2501.14685/

## Key facts

- Cited by 5 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 24 Jan 2025.

## Built on by (most cited first, 5 of 5)

- [Stochastic Entanglement Configuration for Constructive Entanglement Topologies in Quantum Machine Learning with Application to Cardiac MRI](https://mlsanity.com/paper/2507.11401/) (2025)
- [Benchmarking PNW Model for MedMNIST to 100% Accuracy](https://mlsanity.com/paper/2604.18916/) (2026)
- [CNNs, Transformers, Hybrid, and Vision Language Models for Skin Cancer Detection](https://mlsanity.com/paper/2605.26294/) (2026)
- [A Shared-Backbone Approach for Multi-Task MedMNIST Classification](https://mlsanity.com/paper/2609.06838/) (2026)
- [Beyond the Encoder: Object-Centric Representations Emerging from Segmentation Foundation Models](https://mlsanity.com/paper/2610.18543/) (2026)

## Abstract (arXiv)

Foundation models are widely employed in medical image analysis, due to their high adaptability and generalizability for downstream tasks. With the increasing number of foundation models being released, model selection has become an important issue. In this work, we study the capabilities of foundation models in medical image classification tasks by conducting a benchmark study on the MedMNIST dataset. Specifically, we adopt various foundation models ranging from convolutional to Transformer-based models and implement both end-to-end training and linear probing for all classification tasks. The results demonstrate the significant potential of these pre-trained models when transferred for medical image classification. We further conduct experiments with different image sizes and various sizes of training data. By analyzing all the results, we provide preliminary, yet useful insights and conclusions on this topic.