# ProtoMedX: Towards Explainable Multi-Modal Prototype Learning for Bone Health Classification

arXiv: https://arxiv.org/abs/2509.14830 · HTML: https://mlsanity.com/paper/2509.14830/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 13 of its 33 references that are in the map.
- First posted on arXiv 18 Sep 2025.

## Builds on (most cited first, 13 of 13)

- [A Simple Framework for Contrastive Learning of Visual Representations](https://mlsanity.com/paper/2002.05709/) (2020)
- [Grad-CAM: Visual Explanations from Deep Networks via Gradient-based Localization](https://mlsanity.com/paper/1610.02391/) (2016)
- [A Unified Approach to Interpreting Model Predictions](https://mlsanity.com/paper/1705.07874/) (2017)
- [Visualizing and Understanding Convolutional Networks](https://mlsanity.com/paper/1311.2901/) (2013)
- [CrossViT: Cross-Attention Multi-Scale Vision Transformer for Image Classification](https://mlsanity.com/paper/2103.14899/) (2021)
- [This Looks Like That: Deep Learning for Interpretable Image Recognition](https://mlsanity.com/paper/1806.10574/) (2018)
- [The Disagreement Problem in Explainable Machine Learning: A Practitioner's Perspective](https://mlsanity.com/paper/2202.01602/) (2022)
- [Towards Explainable Deep Neural Networks (xDNN)](https://mlsanity.com/paper/1912.02523/) (2019)
- [Do Not Trust Additive Explanations](https://mlsanity.com/paper/1903.11420/) (2019)
- [Semi-Supervised Learning for Bone Mineral Density Estimation in Hip X-ray Images](https://mlsanity.com/paper/2103.13482/) (2021)
- [Augment like there's no tomorrow: Consistently performing neural networks for medical imaging](https://mlsanity.com/paper/2206.15274/) (2022)
- [Analyzing Data Augmentation for Medical Images: A Case Study in Ultrasound Images](https://mlsanity.com/paper/2403.09828/) (2024)
- [A Critical Appraisal of Data Augmentation Methods for Imaging-Based Medical Diagnosis Applications](https://mlsanity.com/paper/2301.02181/) (2022)

## Built on by (most cited first, 2 of 2)

- [ProtoMedAgent: Multimodal Clinical Interpretability via Privacy-Aware Agentic Workflows](https://mlsanity.com/paper/2605.14113/) (2026)
- [T-DuMpRa: Teacher-guided Dual-path Multi-prototype Retrieval Augmented framework for fine-grained medical image classification](https://mlsanity.com/paper/2604.17360/) (2026)

## Abstract (arXiv)

Bone health studies are crucial in medical practice for the early detection and treatment of Osteopenia and Osteoporosis. Clinicians usually make a diagnosis based on densitometry (DEXA scans) and patient history. The applications of AI in this field are ongoing research. Most successful methods rely on deep learning models that use vision alone (DEXA/X-ray imagery) and focus on prediction accuracy, while explainability is often disregarded and left to post hoc assessments of input contributions. We propose ProtoMedX, a multi-modal (multimodal) model that uses both DEXA scans of the lumbar spine and patient records. ProtoMedX's prototype-based architecture is explainable by design, which is crucial for medical applications, especially in the context of the upcoming EU AI Act, as it allows explicit analysis of model decisions, including incorrect ones. ProtoMedX demonstrates state-of-the-art performance in bone health classification while also providing explanations that can be visually understood by clinicians. Using a dataset of 4,160 real NHS patients, the proposed ProtoMedX achieves 87.58% accuracy in vision-only tasks and 89.8% in its multi-modal variant, both surpassing existing published methods.