# Does Bigger Mean Better? Comparitive Analysis of CNNs and Biomedical Vision Language Modles in Medical Diagnosis

arXiv: https://arxiv.org/abs/2510.00411 · HTML: https://mlsanity.com/paper/2510.00411/

## Key facts

- Cited by 9 papers in the map (data through 14 Oct 2026).
- Builds on 14 of its 34 references that are in the map.
- First posted on arXiv 1 Oct 2025.

## Builds on (most cited first, 14 of 14)

- [Learning Transferable Visual Models From Natural Language Supervision](https://mlsanity.com/paper/2103.00020/) (2021)
- [A Survey on Deep Learning in Medical Image Analysis](https://mlsanity.com/paper/1702.05747/) (2017)
- [MedMNIST v2 -- A large-scale lightweight benchmark for 2D and 3D biomedical image classification](https://mlsanity.com/paper/2110.14795/) (2021)
- [PMC-CLIP: Contrastive Language-Image Pre-training using Biomedical Documents](https://mlsanity.com/paper/2303.07240/) (2023)
- [DepthSSC: Monocular 3D Semantic Scene Completion via Depth-Spatial Alignment and Voxel Adaptation](https://mlsanity.com/paper/2311.17084/) (2023)
- [Swift Sampler: Efficient Learning of Sampler by 10 Parameters](https://mlsanity.com/paper/2410.05578/) (2024)
- [AMMKD: Adaptive Multimodal Multi-teacher Distillation for Lightweight Vision-Language Models](https://mlsanity.com/paper/2509.00039/) (2025)
- [MemeBLIP2: A novel lightweight multimodal system to detect harmful memes](https://mlsanity.com/paper/2504.21226/) (2025)
- [Applications of Large Models in Medicine](https://mlsanity.com/paper/2502.17132/) (2025)
- [Selective Attention Federated Learning: Improving Privacy and Efficiency for Clinical Text Classification](https://mlsanity.com/paper/2504.11793/) (2025)
- [Modeling Parkinson's Disease Progression Using Longitudinal Voice Biomarkers: A Comparative Study of Statistical and Neural Mixed-Effects Models](https://mlsanity.com/paper/2507.20058/) (2025)
- [Physics-Informed Neural Networks for Enhanced Interface Preservation in Lattice Boltzmann Multiphase Simulations](https://mlsanity.com/paper/2504.10539/) (2025)
- [LBM-GNN: Graph Neural Network Enhanced Lattice Boltzmann Method](https://mlsanity.com/paper/2504.14494/) (2025)
- [Markup Language Modeling for Web Document Understanding](https://mlsanity.com/paper/2509.20940/) (2025)

## Built on by (most cited first, 9 of 9)

- [CEC-Zero: Zero-Supervision Character Error Correction with Self-Generated Rewards](https://mlsanity.com/paper/2512.23971/) (2025)
- [Self-Paced Learning for Images of Antinuclear Antibodies](https://mlsanity.com/paper/2511.21519/) (2025)
- [Seeing through the Conflict: Transparent Knowledge Conflict Handling in Retrieval-Augmented Generation](https://mlsanity.com/paper/2601.06842/) (2026)
- [Renaissance of RNNs in Streaming Clinical Time Series: Compact Recurrence Remains Competitive with Transformers](https://mlsanity.com/paper/2510.16677/) (2025)
- [Lightweight Baselines for Medical Abstract Classification: DistilBERT with Cross-Entropy as a Strong Default](https://mlsanity.com/paper/2510.10025/) (2025)
- [Exploring Accurate and Transparent Domain Adaptation in Predictive Healthcare via Concept-Grounded Orthogonal Inference](https://mlsanity.com/paper/2602.12542/) (2026)
- [Beyond More Context: Retrieval Diversity Boosts Multi-Turn Intent Understanding](https://mlsanity.com/paper/2510.17940/) (2025)
- [Physiologically Informed Deep Learning: A Multi-Scale Framework for Next-Generation PBPK Modeling](https://mlsanity.com/paper/2602.18472/) (2026)
- [Layout-Guided Masking for GROBID: Lightweight Structural Gains in Large-Scale Scientific PDF Ingestion](https://mlsanity.com/paper/2609.26381/) (2026)

## Abstract (arXiv)

The accurate interpretation of chest radiographs using automated methods is a critical task in medical imaging. This paper presents a comparative analysis between a supervised lightweight Convolutional Neural Network (CNN) and a state-of-the-art, zero-shot medical Vision-Language Model (VLM), BiomedCLIP, across two distinct diagnostic tasks: pneumonia detection on the PneumoniaMNIST benchmark and tuberculosis detection on the Shenzhen TB dataset. Our experiments show that supervised CNNs serve as highly competitive baselines in both cases. While the default zero-shot performance of the VLM is lower, we demonstrate that its potential can be unlocked via a simple yet crucial remedy: decision threshold calibration. By optimizing the classification threshold on a validation set, the performance of BiomedCLIP is significantly boosted across both datasets. For pneumonia detection, calibration enables the zero-shot VLM to achieve a superior F1-score of 0.8841, surpassing the supervised CNN's 0.8803. For tuberculosis detection, calibration dramatically improves the F1-score from 0.4812 to 0.7684, bringing it close to the supervised baseline's 0.7834. This work highlights a key insight: proper calibration is essential for leveraging the full diagnostic power of zero-shot VLMs, enabling them to match or even outperform efficient, task-specific supervised models.