# Reproducible evaluation of classification methods in Alzheimer's disease: framework and application to MRI and PET data

arXiv: https://arxiv.org/abs/1808.06452 · HTML: https://mlsanity.com/paper/1808.06452/

## Key facts

- Cited by 14 papers in the map (data through 14 Oct 2026).
- First posted on arXiv 20 Aug 2018.

## Built on by (most cited first, 14 of 14)

- [Automatic quality control of brain T1-weighted magnetic resonance images for a clinical data warehouse](https://mlsanity.com/paper/2104.08131/) (2021)
- [Reproducibility in machine learning for medical imaging](https://mlsanity.com/paper/2209.05097/) (2022)
- [AXIAL: Attention-based eXplainability for Interpretable Alzheimer's Localized Diagnosis using 2D CNNs on 3D MRI brain scans](https://mlsanity.com/paper/2407.02418/) (2024)
- [Latent Diffusion Autoencoders: Toward Efficient and Meaningful Unsupervised Representation Learning in Medical Imaging](https://mlsanity.com/paper/2504.08635/) (2025)
- [U-PET: MRI-based Dementia Detection with Joint Generation of Synthetic FDG-PET Images](https://mlsanity.com/paper/2206.08078/) (2022)
- [From Group-Differences to Single-Subject Probability: Conformal Prediction-based Uncertainty Estimation for Brain-Age Modeling](https://mlsanity.com/paper/2302.05304/) (2023)
- [Generative models of MRI-derived neuroimaging features and associated dataset of 18,000 samples](https://mlsanity.com/paper/2407.12897/) (2024)
- [PHGNN: A Novel Prompted Hypergraph Neural Network to Diagnose Alzheimer's Disease](https://mlsanity.com/paper/2503.14577/) (2025)
- [An explainable two-dimensional single model deep learning approach for Alzheimer's disease diagnosis and brain atrophy localization](https://mlsanity.com/paper/2107.13200/) (2021)
- [AI for the prediction of early stages of Alzheimer's disease from neuroimaging biomarkers -- A narrative review of a growing field](https://mlsanity.com/paper/2406.17822/) (2024)
- [Automatic rating of incomplete hippocampal inversions evaluated across multiple cohorts](https://mlsanity.com/paper/2408.02496/) (2024)
- [Hierarchical Variable Importance with Statistical Control for Medical Data-Based Prediction](https://mlsanity.com/paper/2508.08724/) (2025)
- [Cross-Dataset Linkage of Brain MRI using Image Similarity Measures](https://mlsanity.com/paper/2602.10043/) (2026)
- [Cross-Modal Knowledge Distillation for PET-Free Amyloid-Beta Detection from MRI](https://mlsanity.com/paper/2604.12574/) (2026)

## Abstract (arXiv)

A large number of papers have introduced novel machine learning and feature extraction methods for automatic classification of AD. However, they are difficult to reproduce because key components of the validation are often not readily available. These components include selected participants and input data, image preprocessing and cross-validation procedures. The performance of the different approaches is also difficult to compare objectively. In particular, it is often difficult to assess which part of the method provides a real improvement, if any. We propose a framework for reproducible and objective classification experiments in AD using three publicly available datasets (ADNI, AIBL and OASIS). The framework comprises: i) automatic conversion of the three datasets into BIDS format, ii) a modular set of preprocessing pipelines, feature extraction and classification methods, together with an evaluation framework, that provide a baseline for benchmarking the different components. We demonstrate the use of the framework for a large-scale evaluation on 1960 participants using T1 MRI and FDG PET data. In this evaluation, we assess the influence of different modalities, preprocessing, feature types, classifiers, training set sizes and datasets. Performances were in line with the state-of-the-art. FDG PET outperformed T1 MRI for all classification tasks. No difference in performance was found for the use of different atlases, image smoothing, partial volume correction of FDG PET images, or feature type. Linear SVM and L2-logistic regression resulted in similar performance and both outperformed random forests. The classification performance increased along with the number of subjects used for training. Classifiers trained on ADNI generalized well to AIBL and OASIS. All the code of the framework and the experiments is publicly available at: https://gitlab.icm-institute.org/aramislab/AD-ML.