# TriFormer: A Multi-modal Transformer Framework For Mild Cognitive Impairment Conversion Prediction

arXiv: https://arxiv.org/abs/2307.07177 · HTML: https://mlsanity.com/paper/2307.07177/

## Key facts

- Cited by 2 papers in the map (data through 14 Oct 2026).
- Builds on 4 of its 12 references that are in the map.
- First posted on arXiv 14 Jul 2023.

## Builds on (most cited first, 4 of 4)

- [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://mlsanity.com/paper/1810.04805/) (2018)
- [An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale](https://mlsanity.com/paper/2010.11929/) (2020)
- [Convolutional Neural Networks for Classification of Alzheimer's Disease: Overview and Reproducible Evaluation](https://mlsanity.com/paper/1904.07773/) (2019)
- [MRI-based Alzheimer's disease prediction via distilling the knowledge in multi-modal data](https://mlsanity.com/paper/2104.03618/) (2021)

## Built on by (most cited first, 2 of 2)

- [Enhancing Multimodal Medical Image Classification using Cross-Graph Modal Contrastive Learning](https://mlsanity.com/paper/2410.17494/) (2024)
- [Sparse Multi-Modal Transformer with Masking for Alzheimer's Disease Classification](https://mlsanity.com/paper/2512.14491/) (2025)

## Abstract (arXiv)

The prediction of mild cognitive impairment (MCI) conversion to Alzheimer's disease (AD) is important for early treatment to prevent or slow the progression of AD. To accurately predict the MCI conversion to stable MCI or progressive MCI, we propose Triformer, a novel transformer-based framework with three specialized transformers to incorporate multi-model data. Triformer uses I) an image transformer to extract multi-view image features from medical scans, II) a clinical transformer to embed and correlate multi-modal clinical data, and III) a modality fusion transformer that produces an accurate prediction based on fusing the outputs from the image and clinical transformers. Triformer is evaluated on the Alzheimer's Disease Neuroimaging Initiative (ANDI)1 and ADNI2 datasets and outperforms previous state-of-the-art single and multi-modal methods.