Unlocking the Power of Medical Tabular Data via Semantic-Aware Multimodal Pre-training
On the map: Multimodal LLMs / vision-language models → Medical vision-language models
Abstract (arXiv)
While vision-language models dominate medical representation learning, unstructured text lacks the dense, quantitative diagnostic phenotypes inherent in structured clinical tables. However, existing multimodal pre-training methods underutilize this potential due to semantic-agnostic designs that treat tabular inputs as flat vectors and employ unstable continuous regression objectives. To overcome this, we propose a novel semantic-aware framework explicitly modeling the intrinsic two-dimensional structure of tabular data. First, addressing the inter-feature hierarchy of varying diagnostic importance, we introduce Importance-Aware Adaptive Masking to construct a label-free curriculum prioritizing salient features. Second, addressing the intra-feature continuity-discreteness duality, we propose a Soft-Label Discretized Module that replaces unstable numerical regression with stable distribution matching, thereby mathematically preserving ordinal relationships. Extensive experiments across large-scale dermatology (SLICE-3D, HOP) and ophthalmology (EyePACS) datasets establish a new state-of-the-art (SOTA), demonstrating exceptional robustness and cross-domain generalizability.
Key facts. No citing papers in the map yet (data through 7 Oct 2026). Placed in Medical vision-language models, one of 16 directions in Multimodal LLMs / vision-language models. Builds on 16 of its 26 references that are in the map. First posted on arXiv 11 Aug 2026.
Builds on
16 of 26 references.
Closest ancestors · 4
Same direction, newest first
- 2026A Vision-Language Foundation Model for Zero-shot Clinical Collaboration and Automated Concept Discovery in Dermatology11 Feb
- 2025Multi-Aspect Knowledge-Enhanced Medical Vision-Language Pretraining with Multi-Agent Data Generation3 Dec
- 2025MAKE: Multi-Aspect Knowledge-Enhanced Vision-Language Pretraining for Zero-shot Dermatological Assessment14 May
- 2025Derm1M: A Million-scale Vision-Language Dataset Aligned with Clinical Ontology Knowledge for Dermatology19 Mar
Other references · 12
- 2018BERT: Pre-training of Deep Bidirectional Transformers for Language Understandingcited by 40,204
- 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 29,915
- 2020A Simple Framework for Contrastive Learning of Visual Representationscited by 11,979
- 2019BioBERT: a pre-trained biomedical language representation model for biomedical text miningcited by 1,935
- 2025DINOv3cited by 1,297
- 2021Revisiting Deep Learning Models for Tabular Datacited by 725
- 2020AdaBins: Depth Estimation using Adaptive Binscited by 596
- 2021SAINT: Improved Neural Networks for Tabular Data via Row Attention and Contrastive Pre-Trainingcited by 310
- 2021SCARF: Self-Supervised Contrastive Learning using Random Feature Corruptioncited by 151
- 2023Best of Both Worlds: Multimodal Contrastive Learning with Tabular and Imaging Datacited by 68
- 2024TIP: Tabular-Image Pre-training for Multimodal Classification with Incomplete Datacited by 32
- 2025Unleashing the Power of Image-Tabular Self-Supervised Learning via Breaking Cross-Tabular Barrierscited by 5
Built on by
Not cited yet.
Similar, not cited
Close in wording, no citation either way
- 2026Schema-Adaptive Tabular Representation Learning with LLMs for Generalizable Multimodal Clinical Reasoning
- 2025LMOD+: A Comprehensive Multimodal Dataset and Benchmark for Developing and Evaluating Multimodal Large Language Models in Ophthalmology
- 2025Distribution-Based Masked Medical Vision-Language Model Using Structured Reports
- 2026Representation-guided in-context learning for medical image interpretation with multimodal large language models
- 2025On the Importance of Text Preprocessing for Multimodal Representation Learning and Pathology Report Generation
- 2026ASAP: Advancing Medical Volumetric Representation Learning with Anatomy-aware Semantically-adaptive Pre-training
- 2025AMF-MedIT: An Efficient Align-Modulation-Fusion Framework for Medical Image-Tabular Data
- 2026MedLayBench-V: A Large-Scale Benchmark for Expert-Lay Semantic Alignment in Medical Vision Language Models