# DeltaRepresentation: Geometry Supervised Representation Learning of Phenotypes via Counterfactual Reasoning for Medical VLMs

arXiv: https://arxiv.org/abs/2610.10286 · HTML: https://mlsanity.com/paper/2610.10286/

## Key facts

- No citing papers in the map yet (data through 7 Oct 2026).
- Placed in Clinical and biomedical applications, one of 17 directions in LLM reasoning.
- Builds on 7 of its 22 references that are in the map.
- First posted on arXiv 7 Oct 2026.

## Placed in

- LLM reasoning / [Clinical and biomedical applications](https://mlsanity.com/lr/clinical-and-biomedical-applications/)
- Multimodal LLMs / vision-language models / [Medical vision-language models](https://mlsanity.com/mllm/medical-vision-language-models/)

## Builds on (most cited first, 7 of 7)

- [MedGemma Technical Report](https://mlsanity.com/paper/2507.05201/) (2025)
- [M3D: Advancing 3D Medical Image Analysis with Multi-Modal Large Language Models](https://mlsanity.com/paper/2404.00578/) (2024)
- [MedKLIP: Medical Knowledge Enhanced Language-Image Pre-Training in Radiology](https://mlsanity.com/paper/2301.02228/) (2023)
- [MAIRA-2: Grounded Radiology Report Generation](https://mlsanity.com/paper/2406.04449/) (2024)
- [Hulu-Med: A Transparent Generalist Model towards Holistic Medical Vision-Language Understanding](https://mlsanity.com/paper/2510.08668/) (2025)
- [MedRAX: Medical Reasoning Agent for Chest X-ray](https://mlsanity.com/paper/2502.02673/) (2025)
- [MedGemma 1.5 Technical Report](https://mlsanity.com/paper/2604.05081/) (2026)

## Abstract (arXiv)

Medical vision-language models (VLMs) have shown increasing potential for radiological image interpretation. Medical VLMs encode radiological images into visual representations that capture both anatomical and phenotypic information for diagnosis. Existing approaches improve pathological phenotype representations through semantic-guided representation alignment. However, pathological phenotypes arise as lesion-specific visual changes superimposed on underlying normal anatomy. Such semantic alignment approaches fail to model the phenotype-specific increment relative to the corresponding normal anatomical representation. To address this gap, we propose DeltaRepresentation, a visual phenotype representation learning framework based on counterfactual reasoning for medical VLMs. It comprises BaseAnatomy, a geometry-supervised representation learning module, and DeltaPhenotype, a counterfactual incremental representation learning module. BaseAnatomy provides fine-grained geometric supervision through spatial relationships across and within anatomical structures. DeltaPhenotype computes the representation increment between lesion representations and their corresponding normal anatomical representations, and supervises increments associated with the same phenotype to cluster in the representation space. Experiments on textitReXGroundingCT and textitLIDC-IDRI demonstrate that DeltaRepresentation effectively structures pathological phenotype representations and improves lesion grounding and phenotype characterization accuracy in medical VLMs. Code is available at https://anonymous.4open.science/r/deltarep-CF6D.