# Combining 3D Image and Tabular Data via the Dynamic Affine Feature Map Transform

arXiv: https://arxiv.org/abs/2107.05990 · HTML: https://mlsanity.com/paper/2107.05990/

## Key facts

- Cited by 38 papers in the map (data through 14 Oct 2026).
- Builds on 8 of its 32 references that are in the map.
- First posted on arXiv 13 Jul 2021.

## Builds on (most cited first, 8 of 8)

- [Deep Residual Learning for Image Recognition](https://mlsanity.com/paper/1512.03385/) (2015)
- [Squeeze-and-Excitation Networks](https://mlsanity.com/paper/1709.01507/) (2017)
- [FiLM: Visual Reasoning with a General Conditioning Layer](https://mlsanity.com/paper/1709.07871/) (2017)
- [Convolutional Neural Networks for Classification of Alzheimer's Disease: Overview and Reproducible Evaluation](https://mlsanity.com/paper/1904.07773/) (2019)
- [Semi-Structured Deep Piecewise Exponential Models](https://mlsanity.com/paper/2011.05824/) (2020)
- [End-To-End Alzheimer's Disease Diagnosis and Biomarker Identification](https://mlsanity.com/paper/1810.00523/) (2018)
- [A Wide and Deep Neural Network for Survival Analysis from Anatomical Shape and Tabular Clinical Data](https://mlsanity.com/paper/1909.03890/) (2019)
- [INSIDE: Steering Spatial Attention with Non-Imaging Information in CNNs](https://mlsanity.com/paper/2008.10418/) (2020)

## Built on by (most cited first, 30 of 38)

- [Deep Neural Networks and Tabular Data: A Survey](https://mlsanity.com/paper/2110.01889/) (2021)
- [MedFuse: Multi-modal fusion with clinical time-series data and chest X-ray images](https://mlsanity.com/paper/2207.07027/) (2022)
- [DrFuse: Learning Disentangled Representation for Clinical Multi-Modal Fusion with Missing Modality and Modal Inconsistency](https://mlsanity.com/paper/2403.06197/) (2024)
- [Deep Multi-modal Fusion of Image and Non-image Data in Disease Diagnosis and Prognosis: A Review](https://mlsanity.com/paper/2203.15588/) (2022)
- [TIP: Tabular-Image Pre-training for Multimodal Classification with Incomplete Data](https://mlsanity.com/paper/2407.07582/) (2024)
- [Review of multimodal machine learning approaches in healthcare](https://mlsanity.com/paper/2402.02460/) (2024)
- [Dynamic Multimodal Information Bottleneck for Multimodality Classification](https://mlsanity.com/paper/2311.01066/) (2023)
- [Multimodal Contrastive Learning and Tabular Attention for Automated Alzheimer's Disease Prediction](https://mlsanity.com/paper/2308.15469/) (2023)
- [STiL: Semi-supervised Tabular-Image Learning for Comprehensive Task-Relevant Information Exploration in Multimodal Classification](https://mlsanity.com/paper/2503.06277/) (2025)
- [GRANDE: Gradient-Based Decision Tree Ensembles for Tabular Data](https://mlsanity.com/paper/2309.17130/) (2023)
- [A multi-stream convolutional neural network for classification of progressive MCI in Alzheimer's disease using structural MRI images](https://mlsanity.com/paper/2203.01944/) (2022)
- [Addressing Asynchronicity in Clinical Multimodal Fusion via Individualized Chest X-ray Generation](https://mlsanity.com/paper/2410.17918/) (2024)
- [CTPD: Cross-Modal Temporal Pattern Discovery for Enhanced Multimodal Electronic Health Records Analysis](https://mlsanity.com/paper/2411.00696/) (2024)
- [Learning Contrastive Multimodal Fusion with Improved Modality Dropout for Disease Detection and Prediction](https://mlsanity.com/paper/2509.18284/) (2025)
- [Medical Diagnosis with Large Scale Multimodal Transformers: Leveraging Diverse Data for More Accurate Diagnosis](https://mlsanity.com/paper/2212.09162/) (2022)
- [Treatment Outcome Prediction for Intracerebral Hemorrhage via Generative Prognostic Model with Imaging and Tabular Data](https://mlsanity.com/paper/2307.12858/) (2023)
- [RegBN: Batch Normalization of Multimodal Data with Regularization](https://mlsanity.com/paper/2310.00641/) (2023)
- [Visual-Attribute Prompt Learning for Progressive Mild Cognitive Impairment Prediction](https://mlsanity.com/paper/2310.14158/) (2023)
- [A Progressive Single-Modality to Multi-Modality Classification Framework for Alzheimer's Disease Sub-type Diagnosis](https://mlsanity.com/paper/2407.18466/) (2024)
- [VT-Bench: A Unified Benchmark for Visual-Tabular Multi-Modal Learning](https://mlsanity.com/paper/2605.08146/) (2026)
- [Survival Analysis for Idiopathic Pulmonary Fibrosis using CT Images and Incomplete Clinical Data](https://mlsanity.com/paper/2203.11391/) (2022)
- [Automatic Prediction of Stroke Treatment Outcomes: Latest Advances and Perspectives](https://mlsanity.com/paper/2412.04812/) (2024)
- [Diffusion Bridge Networks Simulate Clinical-grade PET from MRI for Dementia Diagnostics](https://mlsanity.com/paper/2510.15556/) (2025)
- [TabMixer: Noninvasive Estimation of the Mean Pulmonary Artery Pressure via Imaging and Tabular Data Mixing](https://mlsanity.com/paper/2409.07564/) (2024)
- [PHGNN: A Novel Prompted Hypergraph Neural Network to Diagnose Alzheimer's Disease](https://mlsanity.com/paper/2503.14577/) (2025)
- [Small Lesions-aware Bidirectional Multimodal Multiscale Fusion Network for Lung Disease Classification](https://mlsanity.com/paper/2508.04205/) (2025)
- [AMF-MedIT: An Efficient Align-Modulation-Fusion Framework for Medical Image-Tabular Data](https://mlsanity.com/paper/2506.19439/) (2025)
- [Amplifying Prominent Representations in Multimodal Learning via Variational Dirichlet Process](https://mlsanity.com/paper/2510.20736/) (2025)
- [Cross-Modal Alignment via Variational Copula Modelling](https://mlsanity.com/paper/2511.03196/) (2025)
- [When Does Multimodal Learning Help in Healthcare? A Benchmark on EHR and Chest X-Ray Fusion](https://mlsanity.com/paper/2602.23614/) (2026)

## Abstract (arXiv)

Prior work on diagnosing Alzheimer's disease from magnetic resonance images of the brain established that convolutional neural networks (CNNs) can leverage the high-dimensional image information for classifying patients. However, little research focused on how these models can utilize the usually low-dimensional tabular information, such as patient demographics or laboratory measurements. We introduce the Dynamic Affine Feature Map Transform (DAFT), a general-purpose module for CNNs that dynamically rescales and shifts the feature maps of a convolutional layer, conditional on a patient's tabular clinical information. We show that DAFT is highly effective in combining 3D image and tabular information for diagnosis and time-to-dementia prediction, where it outperforms competing CNNs with a mean balanced accuracy of 0.622 and mean c-index of 0.748, respectively. Our extensive ablation study provides valuable insights into the architectural properties of DAFT. Our implementation is available at https://github.com/ai-med/DAFT.