M2PFN: End-to-End Disentangled Alignment for Generalizable Multimodal In-Context Learning in Alzheimer's Disease
Not on the map. Why?
Abstract (arXiv)
While various multimodal methods combining imaging and tabular data for Alzheimer's disease (AD) diagnosis were proposed, they are often limited in generalization across cohorts. In-context learning (ICL) has demonstrated excellent generalization performances and high flexibility in foundational tabular models such as TabPFN. To extend TabPFN's ICL to multimodal AD analysis, the main obstacle is that TabPFN is meta-trained on synthetic tabular priors that do not naturally match the statistical structure of image-derived features. We propose M2PFN, an end-to-end framework that turns this tabular foundation model into a multimodal AD predictor. M2PFN (i) performs differentiable inference through TabPFN's transformer, back-propagating task gradients into 3D-MRI and tabular encoders; (ii) aligns the two modalities into a shared subspace, via disentanglement and a contrastive objective, matched to the ICL engine's prior; and (iii) folds in a frozen tabular-only prediction through a learnable gated shortcut. Because the ICL engine stays frozen, its in-context mechanism is preserved for test-time generalization, while end-to-end training shapes the encoders into features it can exploit. On ADNI (n=2240, three-class CN/MCI/AD), M2PFN attains 65.55% macro-F1 and 82.21% macro-AUC, surpassing a comprehensive set of unimodal and multimodal baselines. By swapping only the head for a TabPFN regressor, the same architecture regresses baseline MMSE on a 1250-subject sub-cohort to test MAE 1.743, outperforming every multimodal baseline. On two external cohorts (OASIS-3 and SCAN) with no retraining, M2PFN achieves the best AUC and the lowest MMSE MAE across all baselines, and transfers even when the cognitive instrument changes.
Key facts. No citing papers in the map yet (data through 14 Oct 2026). Builds on 29 of its 43 references that are in the map. First posted on arXiv 23 Sep 2026.
Builds on
29 of 43 references.
References · 29
- 2015Deep Residual Learning for Image Recognitioncited by 52,096
- 2021Learning Transferable Visual Models From Natural Language Supervisioncited by 33,744
- 2020Language Models are Few-Shot Learnerscited by 32,419
- 2020An Image is Worth 16x16 Words: Transformers for Image Recognition at Scalecited by 30,186
- 2016Densely Connected Convolutional Networkscited by 6,720
- 2016XGBoost: A Scalable Tree Boosting Systemcited by 5,166
- 2017Squeeze-and-Excitation Networkscited by 4,789
- 2022Flamingo: a Visual Language Model for Few-Shot Learningcited by 4,386
- 2021Prefix-Tuning: Optimizing Continuous Prompts for Generationcited by 3,915
- 2021The Power of Scale for Parameter-Efficient Prompt Tuningcited by 3,795
- 2021Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsitycited by 3,073
- 2021Align before Fuse: Vision and Language Representation Learning with Momentum Distillationcited by 1,619
17 more
- 2021Revisiting Deep Learning Models for Tabular Datacited by 740
- 2021GPT Understands, Toocited by 716
- 2022TabPFN: A Transformer That Solves Small Tabular Classification Problems in a Secondcited by 674
- 2019Multimodal Transformer for Unaligned Multimodal Language Sequencescited by 645
- 2022MONAI: An open-source framework for deep learning in healthcarecited by 448
- 2020AutoGluon-Tabular: Robust and Accurate AutoML for Structured Datacited by 383
- 2021Transformers Can Do Bayesian Inferencecited by 360
- 2019Med3D: Transfer Learning for 3D Medical Image Analysiscited by 288
- 2021Self-Attention Between Datapoints: Going Beyond Individual Input-Output Pairs in Deep Learningcited by 113
- 2022mmFormer: Multimodal Medical Transformer for Incomplete Multimodal Learning of Brain Tumor Segmentationcited by 95
- 2023TabR: Tabular Deep Learning Meets Nearest Neighbors in 2023cited by 74
- 2024Flex-MoE: Modeling Arbitrary Modality Combination via the Flexible Mixture-of-Expertscited by 70
- 2024MoE++: Accelerating Mixture-of-Experts Methods with Zero-Computation Expertscited by 56
- 2021Combining 3D Image and Tabular Data via the Dynamic Affine Feature Map Transformcited by 38
- 2025I2MoE: Interpretable Multimodal Interaction-aware Mixture-of-Expertscited by 30
- 2026MultiModalPFN: Extending Prior-Data Fitted Networks for Multimodal Tabular Learningcited by 10
- 2024Intrinsic User-Centric Interpretability through Global Mixture of Expertscited by 9
Built on by
Not cited yet.